Well, after a long while of testing and head scratching I finally found what
was causing the problem.
We have two desktops in our lab, one running Windows XP and one running Windows
7. Both have identical network cards (Intel CT cards). However, when they
were plugged into a certain switch, one would send data out at the rate
expected, while the Windows 7 box would not. I moved everything to a small
Netgear switch and problem solved. Seems a bit interesting that a high-end
16-port switch would cause issues like that.
So, no problems now with receiving data at rate.
--
Jonathan R. Haws
Electrical Engineer
Space Dynamics Laboratory
(435) 713-3489
jh...@sdl.usu.edu
________________________________________
From: Jonathan Haws [jonathan.h...@sdl.usu.edu]
Sent: Thursday, May 26, 2011 14:00
To: e1000-devel@lists.sourceforge.net
Subject: [E1000-devel] Unicast v. Multicast with e1000e Driver - Dropping
packets problem
All,
I am not sure if this is the right list for this, but I guess this is a good
place to start.
I am developing for an embedded system (a 64-bit Core 2 Duo). I am running a
vanilla kernel (2.6.39). Our application is simply this - receive multicast
data from the network and dump it to an SSD.
My problem comes in two parts: unicast data works somewhat and multicast data
does not at all.
Here is my test scenario (everything is UDP - no TCP):
A PC running Win7 blasts a file over the network - this file simple contains a
ramp of integers (0, 1, 2, 3, 4, 5, ...). On the receiving side I receive the
datagrams (however large - we are allowing for the IP stack to handle
fragmentation) and check the first value of the ramp that is in that datagram.
When the value of the data in the packet is not what I expect, I print an
error, set my counter to the value of the data, and move on.
When I am unicasting, everything is just peachy until I get to large datagram
sizes. I can run at high rates (up to 30 MiB/s I have tested) and don't lose
anything. I have run with varying datagram lengths; from 5000 bytes to 30000
bytes. Once I start to get into the 15000 to 20000 byte range, things go south
(no offense to anyone from the south :) ). My MTU is set to 9000 on the
transmit and receive side (and yes, my switches handle jumbo frames...).
When I am multicasting, things go bad. Even running at a low rate (say 5
MiB/s) I see missing datagrams. The things that really bothers me is that
these do not show up anywhere. ifconfig does not report anything dropped, and
ethtool -S does not report any errors whatsoever. But it is clear to me that
packets are being missed. With a datagram size of 5000 things work okay, but
when I step up to 8500 things go awry.
It seems like something is up with larger datagrams, but I can't seem to put my
finger on it. I have tried both of my network interfaces and they both behave
in the same manner.
I have looked at some sysctl parameters and everything seems to look good. I
increased net.core.rmem_max and net.core.wmem_max to 64 MB (way overkill I
think, nevertheless...). net.ipv4.udp_mem is "186048 248064 372096". That is
just what it is defaulting to, which seems like oodles of pages. Nothing else
I could find really dealt with my scenario.
I have attached the test code if anyone feels like that may help get to the
bottom of the issue. I don't know if this is just an exercise in optimizing
the network stack, or if there really is a bug somewhere. Any way you want to
look at it, any help is appreciated.
Output follows...
Thanks!
Jonathan
Here is the hardware I have:
ADLGS45 (Advanced Digital Logic PCI/104-Express Core 2 Duo SBC)
eth0 = Intel 82567LM-2 Gigabit Network Controller
eth1 = Intel 82574L Gigabit Network Controller
Here is some output from the system:
# ifconfig
eth0 Link encap:Ethernet HWaddr 00:01:05:0A:4A:92
inet addr:172.31.22.90 Bcast:172.31.22.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:1674366 errors:0 dropped:0 overruns:0 frame:0
TX packets:2620 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:10694569545 (9.9 GiB) TX bytes:459481 (448.7 KiB)
Interrupt:20 Memory:fdfc0000-fdfe0000
# ethtool -i eth0
driver: e1000e
version: 1.3.10-k2
firmware-version: 1.8-4
bus-info: 0000:00:19.0
# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
# ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: off
# ethtool -S eth0
NIC statistics:
rx_packets: 1674840
tx_packets: 2772
rx_bytes: 10694622001
tx_bytes: 481113
rx_broadcast: 1063
tx_broadcast: 0
rx_multicast: 159412
tx_multicast: 4
rx_errors: 0
tx_errors: 0
tx_dropped: 0
multicast: 159412
collisions: 0
rx_length_errors: 0
rx_over_errors: 0
rx_crc_errors: 0
rx_frame_errors: 0
rx_no_buffer_count: 0
rx_missed_errors: 0
tx_aborted_errors: 0
tx_carrier_errors: 0
tx_fifo_errors: 0
tx_heartbeat_errors: 0
tx_window_errors: 0
tx_abort_late_coll: 0
tx_deferred_ok: 0
tx_single_coll_ok: 0
tx_multi_coll_ok: 0
tx_timeout_count: 0
tx_restart_queue: 0
rx_long_length_errors: 0
rx_short_length_errors: 0
rx_align_errors: 0
tx_tcp_seg_good: 0
tx_tcp_seg_failed: 0
rx_flow_control_xon: 0
rx_flow_control_xoff: 0
tx_flow_control_xon: 0
tx_flow_control_xoff: 0
rx_long_byte_count: 10694622001
rx_csum_offload_good: 106021
rx_csum_offload_errors: 0
rx_header_split: 0
alloc_rx_buff_failed: 0
tx_smbus: 0
rx_smbus: 0
dropped_smbus: 0
rx_dma_failed: 0
tx_dma_failed: 0
Output for eth1 is very similar, but here is ethtool -i eth1
# ethtool -i eth1
driver: e1000e
version: 1.3.10-k2
firmware-version: 15.255-15
bus-info: 0000:02:00.0
Now here is some output from my test program. Packets = XXX --> MB = XXXX is
just a status thread printing out the current rates.
# benchnetRX
Usage: benchnetRX <ip> <port> <max datagramlen> <recvbuf in KB>
<multicast> <benchmark>
# benchnetRX 3 19601 50000 16384 0 1
-> Beginning network benchmark ...
-> Creating socket for benchmark on 00000000:19601
-> Packets = 159 --> MB = 1.011
-> Packets = 750 --> MB = 4.768
lost packets = 0
-> Packets = 749 --> MB = 4.762
lost packets = 1
-> Packets = 750 --> MB = 4.768
lost packets = 1
-> Packets = 748 --> MB = 4.756
-> Packets = 749 --> MB = 4.762
lost packets = 2
-> Packets = 746 --> MB = 4.743
lost packets = 5
-> Packets = 739 --> MB = 4.698
-> Packets = 520 --> MB = 3.306
^C -> Exiting network RX benchmark...
# benchnetRX 3 19601 50000 16384 1 1
-> Beginning network benchmark ...
-> Creating socket for benchmark on e0010103:19601
-> Packets = 256 --> MB = 1.628
lost packets = 1
-> Packets = 750 --> MB = 4.768
-> Packets = 749 --> MB = 4.762
lost packets = 3
-> Packets = 750 --> MB = 4.768
lost packets = 0
-> Packets = 751 --> MB = 4.775
lost packets = 0
-> Packets = 749 --> MB = 4.762
-> Packets = 750 --> MB = 4.768
lost packets = 3
-> Packets = 750 --> MB = 4.768
lost packets = 2
-> Packets = 743 --> MB = 4.724
-> Packets = 718 --> MB = 4.565
lost packets = 37
^C -> Exiting network RX benchmark...
--
Jonathan R. Haws
Electrical Engineer
Space Dynamics Laboratory
(435) 713-3489
jh...@sdl.usu.edu
------------------------------------------------------------------------------
vRanger cuts backup time in half-while increasing security.
With the market-leading solution for virtual backup and recovery,
you get blazing-fast, flexible, and affordable data protection.
Download your free trial now.
http://p.sf.net/sfu/quest-d2dcopy1
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit
http://communities.intel.com/community/wired
------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit
http://communities.intel.com/community/wired
