> Thats why the flush flag is always set and the GRO stack does > not work at all. I'm willing to dig deeper into this but I'm unsure > if those fields are filled on sender or receiver side and especially > where in the IPoIB stack.
Maybe I got the reason for that strange ack behaviour during large NFS over IPoIB reads and hopefully someone can confirm this If I turn on TSO for an IPoIB datagram interface on the sender side GRO on the receiver side is totally broken. This due to the fact that TSO "generates" large 60k packets that are offloaded into fragments. Each of these fragments has the same ID in the packet header. GRO expects IDs to be in incremental order and issues a flush after each package. Each flush results in an ACK packet back to the server. With TSO disabled GRO can kick in. Packets are build with sequential IDs. GRO only acknowledges every few packets. For a fully cached file read of 6GB the numbers read: TSO on: ~220MByte/s - 1,522,679 MLX4 Interrupts on server TSO off: ~550MByte/s - 318,322 MLX4 Interrupts on server Is there any chance IPoIB TSO handling can be optimized? Markus
**************************************************************************** Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet. Ãber das Internet versandte E-Mails können unter fremden Namen erstellt oder manipuliert werden. Deshalb ist diese als E-Mail verschickte Nachricht keine rechtsverbindliche Willenserklärung. Collogia Unternehmensberatung AG Ubierring 11 D-50678 Köln Vorstand: Kadir Akin Dr. Michael Höhnerbach Vorsitzender des Aufsichtsrates: Hans Kristian Langva Registergericht: Amtsgericht Köln Registernummer: HRB 52 497 This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. e-mails sent over the internet may have been written under a wrong name or been manipulated. That is why this message sent as an e-mail is not a legally binding declaration of intention. Collogia Unternehmensberatung AG Ubierring 11 D-50678 Köln executive board: Kadir Akin Dr. Michael Höhnerbach President of the supervisory board: Hans Kristian Langva Registry office: district court Cologne Register number: HRB 52 497 ****************************************************************************
