Hi again,
I made some testing today in my desktop machine, I won't try it again
in the server until I make some progress in my machine since it is
shared and someone can kill me if I break the network connection again
:).
About the issue playing the file 10 times slower, this only happened
in that shared server with catapulta linux when I ran with the latest
tcpreplay version, 3.4.2. This is the network card if that is of any
help (it is using the e1000 driver):
03:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708
Gigabit Ethernet (rev 12)
Subsystem: Hewlett-Packard Company NC373i Integrated Multifunction
Gigabit Server Adapter
Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 16
Memory at f8000000 (64-bit, non-prefetchable) [size=32M]
[virtual] Expansion ROM at d1200000 [disabled] [size=2K]
Capabilities: [40] PCI-X non-bridge device
Capabilities: [48] Power Management version 2
Capabilities: [50] Vital Product Data
Capabilities: [58] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0
Enable-
Using the desktop machine I had no luck. These are the stats for
running the original SCTP file at the original speed. Playing the file
at a fixed rate usgin the --mbps option doesn't help.
Actual: 97337 packets (45000654 bytes) sent in 136.50 seconds
Rated: 329675.1 bps, 2.52 Mbps, 713.09 pps
Then I generated a tcp file and played it (instead of my SCTP one) at
original speed. The network connection behaved the same, after some
time it was gone. These are the stats running this TCP file at
original speed:
Actual: 47596 packets (49788435 bytes) sent in 17.79 seconds
Rated: 2798675.5 bps, 21.35 Mbps, 2675.44 pps
Setting manually the timing to random speeds in both files made no
difference, the network connection goes dead too, just as if I play
the file at original speed. I also looked for strange kernel messages
in the log but I found nothing. And more strangely, I am positive the
switch is not causing the problem. I played the file from one machine
to another without any switch, just a cable connecting them, and the
network interface of the machine playing the file is gone in the same
way. I ran out of ideas or things to check, any advise?
This is the network card and driver in this Desktop machine I'm using
to test right now:
00:19.0 Ethernet controller: Intel Corporation 82567LM-3 Gigabit
Network Connection (rev 02)
Subsystem: Dell Device 027f
Flags: bus master, fast devsel, latency 0, IRQ 2299
Memory at febe0000 (32-bit, non-prefetchable) [size=128K]
Memory at febd9000 (32-bit, non-prefetchable) [size=4K]
I/O ports at ecc0 [size=32]
Capabilities: [c8] Power Management version 2
Capabilities: [d0] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0
Enable+
Capabilities: [e0] PCIe advanced features <?>
Kernel driver in use: e1000e
Kernel modules: e1000e
Thanks,
Faustino.
On Sun, Jun 7, 2009 at 23:22, Faustino Frechilla<frechi...@gmail.com> wrote:
> Hi again and thanks for the quick answer!
>
> My anwers inline...
>
> On Fri, Jun 5, 2009 at 21:13, Aaron Turner<synfina...@gmail.com> wrote:
>> First, thanks for the detailed information- it really helps!
>>
>> Anyways, inline...
>>
>> On Fri, Jun 5, 2009 at 12:29 PM, Faustino Frechilla<frechi...@gmail.com>
>> wrote:
>> [snip]
>>
>>> This is the description: I play the pcap file through one network
>>> interface. After some time playing (can't be sure of the actual time)
>>> the network interface is gone for any other requests, that is, the
>>> file is being played, but if I tried to access the server, ping, ssh
>>> or whatever, I got no answers. All the ping packets I send are lost,
>>> so I can't stop tcpreplay and it keeps on running until the file is
>>> finished. An easy way to fix MY _problem_ would be to have an extra
>>> network interface to access the machine while the other one is used to
>>> pump the file, but still I would like to know if this behaviour is
>>> expected or it is something strange that is happening because of the
>>> way tcpreplay tries to simulate the real speed in which the file was
>>> recorded.
>>
>> It's not expected or normal.
>
> I forgot to say in the previous email that the pcap file is made of
> ethernet + ip + sctp packets. It has no TCP or UDP on it, but
> tcpreplay is able to send the packets over the network. This might be
> interesting and hopefully help a little bit.
>
>>
>>> I'm sending the file through no routers. The two hosts are in the same
>>> network. And I'm positive tcpplayer is sending the file because the
>>> other end is reachable and it is receiving the data as expected. No
>>> data is missing in the receiver.
>>
>> Interesting. I assume your tcpreplay box is connected to the other
>> hosts via a switch? Do you know what kind?
>>
>
> Both hosts (the one I play the file from and the one that receives it)
> are connected to a router but they belong to the same network. My
> desktop is in a different network and connects to them through the
> router. I know the network is alive because I can ping the host that
> receives the packets, but not the one that sends the packets.
>
> If I play the file from my desktop to a host in the other network any
> other machine in my network (the network of my desktop) can ping the
> host in the other network but my network interface is gone and I can't
> ping any host or surf the internet.
>
> About the kind of switch I can't know the model until tomorrow, I'll
> try to get back to the list as soon as I can.
>
>>> This is the way I generated the files pcap:
>>>
>>> # I want all the traffic from the network 10. to be pumped into the
>>> destination ip address
>>> tcpprep --regex="10\..*" --pcap=orig.pcap --cachefile=cache.cache
>>
>> FYI, --cidr=10/8 is faster then --regex and does the same thing.
>>
>>> # 172.30.2.65 is the origin and 172.30.2.60 the destination
>>> tcprewrite --endpoints=172.30.2.65:172.30.2.60 --cachefile=cache.cache
>>> --infile=orig.pcap --outfile=temp.pcap --skipbroadcast
>>>
>>> # 00:00:00:00:00:AA and 00:00:00:00:00:BB are the destination and
>>> source mac addresses
>>> tcprewrite --enet-dmac=00:00:00:00:00:AA --enet-smac=00:00:00:00:00:BB
>>> --cachefile=cache.cache --infile=temp.pcap --outfile=to_send.pcap
>>
>> FYI, you could combine the last two commands. :)
>
> Thanks, That will save me a lot of time :)
>
>>
>>> # and this is the offending command
>>> sudo tcpreplay --intf1=eth0 --cachefile=cache.cache to_send.pcap
>>
>> Uh, that last command works? That's a bug. You should NOT be able to
>> use a cachefile with a single interface. I can't say for certain
>> that's the cause of your problem though.
>
> That was my mistake. You are right, I shouldn't be using the .cache
> file because I don't need it, all I need is tcpreplay to play the file
> through only one interface, no need to split the traffic. Again, I'll
> get back to the list tomorrow either if that solves the problem or not
> because I've got no access to the file or servers now.
>
>>
>>> I tried this very same commands using:
>>>
>>> tcpreplay version 3.3.1 under ubuntu desktop 8.10 in a 2Gb intel core
>>> duo (2 usable processors)
>>> tcpreplay version 3.0.1 under catapulta linux
>>> (http://www.catapulta.org/). From their webpage, Catapulta is an
>>> Ubuntu-based server platform that is designed and packaged
>>> specifically for high-performance network monitoring applications
>>> using commodity hardware. This is a server 32Gb RAM, intel 2 quad core
>>> (2x4 processors)
>>
>> Both versions are a bit long in the tooth, but I'm not aware of any
>> "fix" that would help.
>
> I was using those versions because they are the ones that the ubuntu
> distribution installs: 3.3.1 in ubuntu 8.10 and 3.0.1 in catapulta
> (which is based in ubuntu 7.10)
>
>>
>>> In both cases the file was played at the expected speed (2.5 Mbits/s
>>> aprox.) but after some time the network interface is not reachable (in
>>> both cases)
>>>
>>> Then I compiled pcapplayer 3.4.2 (with pcap 0.9.5) in that catapulta
>>> linux, and played the file. This time the network took more time to be
>>> down and surprisingly tcpreplay is playing the file 10 times slower
>>> than before, so the network is taking 10 times more to come back. The
>>> file is a 10 minutes file so all this issue wasn't a big deal until
>>> now, when I have to wait for 100 minutes instead of 10.
>>
>> Wow, that's really strange. I'd love to get a (compressed) copy of
>> that pcap file- or maybe a few thousand packets worth at least.
>
> I'll try to get you a bit of the file, but there might be some privacy issues
> :(
>
>>
>>> Anyway, I got two questions: Is a normal behavior that the network
>>> interface is gone while I play the file or there is something I'm not
>>> doing fine when I try to play the file? why is tcpplayer 3.4.2 is
>>> playing the file 10 times slower than 3.0.1 under the catapulta linux
>>> box?
>>
>> I don't have a good answer for you to either of those questions. I'm
>> grabbing the catapulta ISO right now to see can get an idea of what is
>> going on. I know certain versions of glibc have been giving my timing
>> code fits, but I thought all those issues were resolved.
>>
>> In the mean time, I'd suggest trying the different timing options and
>> see what happens. Honestly, 2.5Mbps is basically nothing and there's
>> something really strange if you're missing the mark that badly.
>> Also, could you provide me the statistics output at the end of the
>> run? Feel free to run the pcap for a minute and hit Ctrl-C- I don't
>> need a full 10 or 100minute run.
>>
>
> I'll try that tomorrow. I compiled and run tcpplayer 3.4.2 with a file
> generated by 3.0.1. As far as I know that shouldn't make any
> difference but obviously you know much more than me :). Anyway it is
> very interesting that 3.4.2 was playing the file 10 times slower (that
> is a rough calculation, but it is a big coincidence)
>
>>> About the hardware, all I can get now is that the ubuntu desktop 8.10
>>> network card is "00:19.0 Ethernet controller: Intel Corporation
>>> 82567LM-3 Gigabit Network Connection" (I have no access to those
>>> machines at the moment). If anyone needs more information about the
>>> hardware I'll be back to you as soon as I can, but I hope some
>>> questions can be answered roughly in the meantime, I'm dying here out
>>> of curiosity!!
>>
>> I don't have any experience with that chipset, but I've never had
>> these kind of problems with any Intel nic. Do you know what kernel
>> version and if you're tweaking any of the NIC driver settings (usually
>> in /etc/modules.conf). What Intel driver are you using? I think
>> there is an e1000 (older) and e1000e (newer) with more recent kernels.
>>
>
> I'm not aware of any tweaking with the NIC driver, but I'll be 100%
> sure tomorrow. Catapulta linux might have them by default however
> (that is something I'll look into)
>
>> --
>> Aaron Turner
>> http://synfin.net/
>> http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix &
>> Windows
>> Those who would give up essential Liberty, to purchase a little temporary
>> Safety, deserve neither Liberty nor Safety.
>> -- Benjamin Franklin
>>
>> ------------------------------------------------------------------------------
>> OpenSolaris 2009.06 is a cutting edge operating system for enterprises
>> looking to deploy the next generation of Solaris that includes the latest
>> innovations from Sun and the OpenSource community. Download a copy and
>> enjoy capabilities such as Networking, Storage and Virtualization.
>> Go to: http://p.sf.net/sfu/opensolaris-get
>> _______________________________________________
>> Tcpreplay-users mailing list
>> Tcpreplay-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/tcpreplay-users
>> Support Information: http://tcpreplay.synfin.net/trac/wiki/Support
>>
>
------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises
looking to deploy the next generation of Solaris that includes the latest
innovations from Sun and the OpenSource community. Download a copy and
enjoy capabilities such as Networking, Storage and Virtualization.
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
Tcpreplay-users mailing list
Tcpreplay-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/tcpreplay-users
Support Information: http://tcpreplay.synfin.net/trac/wiki/Support
