Hello Brian, I am the developer of mlx4_en driver. I will be happy to help you in this issue, it is not rdma related, so let's take it off line.
Yevgeny Brian Cain wrote: > I'm trying to aggregate 10GbE ports for a realtime data acquisition > application. I need to be able to get consistently high throughput to > satisfy my application's demands. > > I have a dual-X58 host with a MT26448 on each IOH. I can acquire > > 39Gb/s in aggregate across four TCP streams (one per port). This > throughput remains consistent across many consecutive netperf > TCP_STREAM trials, but trying the same experiment hours or days later, > I'll find that the performance suffers (< 30Gb/s). > > irq_balancer is disabled and all of the IRQs' affinity masks for one > card are set to the same core, the other card's IRQs are set to a core > on the other socket. I'm using 9000 byte MTUs. The majority of the > system settings are at their defaults. I'm keeping the task affinity > among the netserver processes static as well. I don't see any error > messages from mlx* in dmesg. > > Can anyone suggest some parameters that might be able to help me find > consistently high performance? > > ~~~~ > $ sudo ethtool -i eth16 > driver: mlx4_en (MT_0BD0110010) > version: 1.4.2 (Sep 24 2009) > firmware-version: 2.6.0 > bus-info: 0000:28:00.0 > > $ lsb_release -a > LSB Version: > core-2.0-noarch:core-3.2-noarch:core-2.0-x86_64:core-3.2-x86_64:desktop-3.2-amd64:desktop-3.2-noarch:graphics-2.0-amd64:graphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch > Distributor ID: SUSE LINUX > Description: openSUSE 11.1 (x86_64) > Release: 11.1 > Codename: n/a > > $ cat /proc/version > Linux version 2.6.29.6-9-debug (r...@z800-revo-0) (gcc version 4.3.2 > [gcc-4_3-branch revision 141291] (SUSE Linux) ) #1 SMP Tue Sep 15 > 11:02:15 CDT 2009 > > $ sudo grep . /sys/module/mlx4_en/parameters/* > /sys/module/mlx4_en/parameters/inline_thold:104 > /sys/module/mlx4_en/parameters/ip_reasm:1 > /sys/module/mlx4_en/parameters/num_lro:32 > /sys/module/mlx4_en/parameters/pfcrx:0 > /sys/module/mlx4_en/parameters/pfctx:0 > /sys/module/mlx4_en/parameters/rss_mask:5 > /sys/module/mlx4_en/parameters/rss_xor:0 > > $ sudo grep . /sys/module/mlx4_core/parameters/* > /sys/module/mlx4_core/parameters/block_loopback:1 > /sys/module/mlx4_core/parameters/debug_level:0 > /sys/module/mlx4_core/parameters/enable_qos:N > /sys/module/mlx4_core/parameters/internal_err_reset:1 > /sys/module/mlx4_core/parameters/log_mtts_per_seg:3 > /sys/module/mlx4_core/parameters/log_num_cq:0 > /sys/module/mlx4_core/parameters/log_num_mac:2 > /sys/module/mlx4_core/parameters/log_num_mcg:0 > /sys/module/mlx4_core/parameters/log_num_mpt:0 > /sys/module/mlx4_core/parameters/log_num_mtt:0 > /sys/module/mlx4_core/parameters/log_num_qp:0 > /sys/module/mlx4_core/parameters/log_num_srq:0 > /sys/module/mlx4_core/parameters/log_num_vlan:0 > /sys/module/mlx4_core/parameters/log_rdmarc_per_qp:0 > /sys/module/mlx4_core/parameters/msi_x:1 > /sys/module/mlx4_core/parameters/set_4k_mtu:0 > /sys/module/mlx4_core/parameters/use_prio:N > > -- > -Brian > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in > the body of a message to [email protected] > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
