> Von: Wendy Cheng [[email protected]] > Gesendet: Donnerstag, 5. September 2013 18:42 > An: Markus Stockhausen > Cc: [email protected]; [email protected] > Betreff: Re: Strange NFS client ACK behaviour > > CC linux-nfs .. maybe this is obvious to someone there ... Two > comments inlined below. > > On Tue, Sep 3, 2013 at 11:28 AM, Markus Stockhausen > <[email protected]> wrote: > > Hello, > > > > we observed a performance drop in our IPoIB NFS backup > > infrastructure since we switched to machines with newer > > kernels. As I do not know where to start I hope someone > > on this list can give me hint where to dig for more details. > > In case of no other reply, I would start w/ a socket program (or a > network performance measuring tool) on the interface that does similar > logic as "dd" you described below; that is, send a 256K message in a > fixed number of loops (so total transfer size somewhere close to your > file size) between client and server, followed by comparing the > interrupt counters (cat /proc/interrtups) on both kernels. If the > interrupt count differs as you described, the problem is most likely > with the IB driver, not NFS layer. > > > > > To make a long story short. We use ConnectX cards with the > > standard kernel drivers on version 2.6.32 (Ubuntu 10.04), 3.5 > > (Ubuntu 12.04) and 3.10 (Fedora 19). The very simple and not > > scientific test consists of mounting a NFS share using IPoIB UD > > network interfaces at MTU of 2044. Afterwards read a large file > > on the client side with dd if=file of=/dev/null bs=256K. > > During the transfer we run a tcpdump on the ibX interface on > > the NFS server side. No special settings for kernel parameters > > until now. > > I don't know much about ConnectX. Not sure what "IPoIB UD" means ? > "Datagram vs. CM" or "TCP vs. UDP" ? >
Hello, thanks for stopping by. I followed your advise and compared the behaviour to a tcpdump of a netserver/netperf session. That reads more normal: 20:06:50.397472 IP server.43845 > client.58489: seq ... length 63744 20:06:50.397531 IP client.58489 > server.43845: ack ... length 0 20:06:50.397567 IP server.43845 > client.58489: seq ... length 63744 20:06:50.397595 IP server.43845 > client.58489: seq ... length 63744 20:06:50.397622 IP server.43845 > client.58489: seq ... length 63744 20:06:50.397632 IP client.58489 > server.43845: ack ... length 0 20:06:50.397667 IP server.43845 > client.58489: seq ... length 63744 20:06:50.397715 IP client.58489 > server.43845: ack ... length 0 20:06:50.397723 IP client.58489 > server.43845: ack ... length 0 It is not really comparable as we se transfers to several ports but at least a much better ratio between send and ack packets. Seems like further digging in NFS behaviour is required. But that may be another story. Markus
**************************************************************************** Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet. Ãber das Internet versandte E-Mails können unter fremden Namen erstellt oder manipuliert werden. Deshalb ist diese als E-Mail verschickte Nachricht keine rechtsverbindliche Willenserklärung. Collogia Unternehmensberatung AG Ubierring 11 D-50678 Köln Vorstand: Kadir Akin Dr. Michael Höhnerbach Vorsitzender des Aufsichtsrates: Hans Kristian Langva Registergericht: Amtsgericht Köln Registernummer: HRB 52 497 This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. e-mails sent over the internet may have been written under a wrong name or been manipulated. That is why this message sent as an e-mail is not a legally binding declaration of intention. Collogia Unternehmensberatung AG Ubierring 11 D-50678 Köln executive board: Kadir Akin Dr. Michael Höhnerbach President of the supervisory board: Hans Kristian Langva Registry office: district court Cologne Register number: HRB 52 497 ****************************************************************************
