The MTU reported by "ifconfig ib0" is the MTU used by the Linux TCP/IP network stack. The MTU reported by ibv_devinfo is the MTU that the hardware is capable of sending. This is limited to 4K by the Infiniband specification. The reason the network stack can have a higher MTU is that ib_ipoib is using the RC QP protocol to send IP messages larger than the hardware MTU. If you use "datagram" mode for ib_ipoib, you will see that the network stack MTU is limited to the hardware MTU - 4.
On Mon, 2008-09-15 at 11:43 -0400, Kumar, Amit H. wrote: > Hello Everyone, > > Why do we see a difference in the MTU reported here by: ibv_devinfo and > "ifconfig ib0" How do we interpret them? > > > Also Is there a document where I can read in detail about IPoIB and > applications that benefit from them. > In general I understand that Socket based applications can make use of IPoIB > for a better bandwidth, thought NOT for a better transport latency. > In short I am trying to understand the difference and advantage, for an > Application using "Ethernet NIC" vs "InfiniBand HCA(IPoIB enabled)", apart > from knowing that there is no advantage in terms of transport latency. > > > Thank you!, > Amit > > # ibv_devinfo > hca_id: mthca0 > fw_ver: 1.2.0 > node_guid: 0006:6a00:9800:e8e7 > sys_image_guid: 0006:6a00:9800:e8e7 > vendor_id: 0x066a > vendor_part_id: 25204 > hw_ver: 0xA0 > board_id: MT_0230000001 > phys_port_cnt: 1 > port: 1 > state: PORT_ACTIVE (4) > max_mtu: 2048 (4) > active_mtu: 2048 (4) > sm_lid: 2 > port_lid: 29 > port_lmc: 0x00 > > > > # ifconfig ib0 > ib0 Link encap:InfiniBand HWaddr > 80:00:04:04:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 > inet addr:172.25.44.250 Bcast:172.25.44.255 Mask:255.255.255.0 > inet6 addr: fe80::206:6a00:a000:e8e7/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:65520 Metric:1 > RX packets:138863 errors:0 dropped:0 overruns:0 frame:0 > TX packets:195994 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:256 > RX bytes:11000200 (10.4 MiB) TX bytes:4321658633 (4.0 GiB) > _______________________________________________ > general mailing list > [email protected] > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
