Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR)
On Tue, Jun 21, 2005 at 04:10:35PM -0700, John Gilmore wrote: I'm picking up the Gigabit Ethernet conversation from a few weeks ago. great :) I'm by no means arguing that the linux network stack is slow. It's just not intended for an application which just wants to get big data streams with low latency and little overhead into userspace. I'm surprised to hear you suggest this (though I note you didn't give numbers, just generalizations like enormous). I didn't state (or at leat want to state) that it was impossible to achieve multiple-gigabit data rates with the Linux network (or even TCP/IP) stack. I just think that it is a horrible waste of cpu cycles and memory bandwidth, when all you want is to get large amounts of data from a local (cloe to the computer) device into an userspace application. This is especially true if your userspace application is a cpu hog like digital signal processing algorithms ;) Ethernet. The last time I looked at the Linux network stack was when Dave Miller was cutting his teeth on the SPARC Linux port and making 100MB/s Ethernet run within a few % of theoretical maximum (he quit when he exceeded 10MBytes/sec over TCP from user process to user process). I had of course expected that somebody would do the same thing for Gigabit Ethernet. This appears to have been done; GigE cards seem to be able to push large amounts of data under Linux. sure. no problem with that. What thruput do YOU see using a good GigE card under Linux? Can you move 100 Mbytes/sec from user process to user process with TCP, across two GigE interfaces and a GigE switch? If so, it's doing much better than USB2 ever will, and much better than any version of FireWire. I'm usually not doing client-to-server TCP benchmarks over my test networks, since I'm more interested in small-packet forwarding performance. I'm sure with the right tuning (and probably some of the recent work on TCP BIC etc.) you can saturate a 1000-base-tx link with a single tcp flow, no question on that. But at what expense? you copy data back and forth between different address spaces (kernel/user), you have lots of complex code that deals with retransmissions, protocol demultiplex, ... that is totally unneccessarry for the gnuradio/USRP kind of application. This msg says that with a 10GB/s Ethernet card (note: 10x as fast as GigE) they can send UDP at 1.9GBps, and send TCP at 2.2Gbps from Chicago to Switzerland. Clearly the Linux kernel overhead isn't keeping thruput below a gigabit: http://www-iepm.slac.stanford.edu/monitoring/bulk/10ge/ No. But in those tests, transferring data is the _real_ application to which you want to dedicate all resources. For USRP, the data transfer is merely a small step, a 'necessarry prerequisite'. There is a lot of research about how to make TCP run at gigabit speeds over *long distances*, particularly in recovering from dropped packets. yes, I'm very well of what's going on in this area (especially since the first results of this work are already merged in recent 2.6.x mainline kernels). Harald, in what way is your experience different? Hope my email clarifies :) -- - Harald Welte [EMAIL PROTECTED] http://gnumonks.org/ Privacy in residential applications is a desirable marketing option. (ETSI EN 300 175-7 Ch. A6) signature.asc Description: Digital signature ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR)
Harald Welte wrote: On Tue, Jun 21, 2005 at 04:10:35PM -0700, John Gilmore wrote: I'm picking up the Gigabit Ethernet conversation from a few weeks ago. great :) I didn't state (or at leat want to state) that it was impossible to achieve multiple-gigabit data rates with the Linux network (or even TCP/IP) stack. I just think that it is a horrible waste of cpu cycles and memory bandwidth, when all you want is to get large amounts of data from a local (cloe to the computer) device into an userspace application. This is especially true if your userspace application is a cpu hog like digital signal processing algorithms ;) I'll point out the obvious here. If it's purely-local, then there's no need to deal with the TCP/IP stack--raw ethernet frames would work just fine, and cut-out much of the processing you're complaining about. I'd configure things with a 1GigE card for the radio data grabber, and another card for my real network traffic. -- Marcus LeechMail: Dept 1A12, M/S: 04352P16 Security Standards AdvisorPhone: (ESN) 393-9145 +1 613 763 9145 Advanced Technology Research Nortel Networks [EMAIL PROTECTED] ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR)
On Wed, Jun 22, 2005 at 12:40:06PM +0200, Harald Welte wrote: On Tue, Jun 21, 2005 at 04:10:35PM -0700, John Gilmore wrote: I'm picking up the Gigabit Ethernet conversation from a few weeks ago. Me too. Please, add your inputs to the following page: http://comsec.com/wiki?USRPnotUSB I'm by no means arguing that the linux network stack is slow. It's just not intended for an application which just wants to get big data streams with low latency and little overhead into userspace. I'm surprised to hear you suggest this (though I note you didn't give numbers, just generalizations like enormous). I didn't state (or at leat want to state) that it was impossible to achieve multiple-gigabit data rates with the Linux network (or even TCP/IP) stack. I just think that it is a horrible waste of cpu cycles and memory bandwidth, when all you want is to get large amounts of data from a local (cloe to the computer) device into an userspace application. This is especially true if your userspace application is a cpu hog like digital signal processing algorithms ;) This is were everybody does not agree. Several of us (most likely hams) want to get large amounts of data from a *distant* (away from the computer) device. IOW, I don't want to get my SHF signal through a long, expensive and lossy cable, while I could have it sampled right at the antenna on top of the pole, and data transported through Ethernet on a cheap CAT 5+ cable. What thruput do YOU see using a good GigE card under Linux? Can you move 100 Mbytes/sec from user process to user process with TCP, across two GigE interfaces and a GigE switch? If so, it's doing much better than USB2 ever will, and much better than any version of FireWire. I'm usually not doing client-to-server TCP benchmarks over my test networks, since I'm more interested in small-packet forwarding performance. I'm sure with the right tuning (and probably some of the recent work on TCP BIC etc.) you can saturate a 1000-base-tx link with a single tcp flow, no question on that. First of all, we have to forget about TCP. If you have to resend, it'll be too late. Besides, implementing TCP in a FPGA or anything that must sustain 1Gbps is no piece of cake. I would rather argue in favor of either raw Ethernet frames or more likely, UDP+RTP. UDP/RTP is easy to handle in a FPGA (almost dumb header), can be routed by IP routers if need be, and RTP has interesting timestamp support plus re-odering if need be. Now talking about small-packet forwarding, this is not going to be the case with a NetSRP, unless you want really low latency. Small-packets are really knee-bending, and people achieving good performance with them deserve respect. However, if we're going to fill up a 1Gbps pipe, there's no point in having small packets. The bigger the packet, the better. For those affraid by the latency, just think of what the OS will add up. USB too already suffers from it, and from the kernel/user fence. Of course, we would not be the first ones to do that, professional video is already streamed like that, from what I can see every day. -- Stephane ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
[Discuss-gnuradio] determining throughput
Hello,
I am trying to determine what the throughput is with my
configuration. I am trying to accomplish this by inputting
a sinusoid and writing it to a file. Having never dealt with
binary files, I am now stuck on reading in values from this
file to see if there are any discontinuities. Below is my
attempt at trying to read in the values (pretty much copied from the
gr_file_source.cc) which gives a segfault. Once I can read in and
access the values I plan to check them for continuity in chunks.
Pointers on whether or not this is the right approach or
if there is a much simpler answer to my question would be
greatly appreciated. Thanks and let me know if there is any
information that would help you help me ; )
// trying to read values in from binary file
// and see if the values in the file are
// continuos in order to determine the
// throughput of my setup
#include iostream
#include fstream
#include vector
#include cstdio
#include sys/types.h
#include sys/stat.h
#include fcntl.h
#include stdexcept
using namespace std;
bool continous() { return false; } //in progress
int main () {
int fd;
const char *filename = output.data;
void * d_fp;
if ((fd = open(filename, ios::binary | ios::ate)) 0){
perror (filename);
throw std::runtime_error (can't open file);
}
if ((d_fp = fdopen(fd, rb)) == NULL){
perror (filename);
throw std::runtime_error (can't open file);
}
vectorvoid * buffer;
char *o = (char *) buffer[0];
int size = 100;
int i;
while (size) {
i = fread(o, 8, size, (FILE *) d_fp);
//size -= i;
size = 0;
o += i * 8;
if (size == 0) // done
break;
if (i 0) // short read try again
continue;
}
fclose ((FILE *) d_fp);
return 0;
}
John Daniel
Virginia Tech
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
[Discuss-gnuradio] SIGBUS in testing gr_vmcircbuf_mmap_shm_open_factory
fyi Offered as a data point for this platform, for contemporary users or those of posterity... Building from CVS for this platform Linux cq 2.6.11-gentoo-r11 #3 Sat Jun 18 20:11:08 PDT 2005 i686 Pentium III (Coppermine) GenuineIntel GNU/Linux gcc v3.4.4 runs an equivalent test $ gdb .libs/lt-test_vmcircbuf $ run -v [...] Testing gr_vmcircbuf_mmap_shm_open_factory... [...] which results in [...] ... test_a_bunch_64x16KB[63] OK Program received signal SIGBUS, Bus error. [Switching to Thread 16384 (LWP 2655)] 0xb7f71c53 in test_a_bunch (factory=0x804a3a0, n=32, size=4194304, start_ptr=0xb5bc, verbose=true) at gr_vmcircbuf.cc:125 125 p[i] = counter + i; The other factory tests (gr_vmcircbuf_sysv_shm_factory, gr_vmcircbuf_mmap_tmpfile_factory) in the suite pass. Mark ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: GigE (was Re: [Discuss-gnuradio] DSP based SDR)
On Wed, Jun 22, 2005 at 12:40:06PM +0200, Harald Welte wrote: On Tue, Jun 21, 2005 at 04:10:35PM -0700, John Gilmore wrote: What thruput do YOU see using a good GigE card under Linux? Can you move 100 Mbytes/sec from user process to user process with TCP, across two GigE interfaces and a GigE switch? If so, it's doing much better than USB2 ever will, and much better than any version of FireWire. I'm usually not doing client-to-server TCP benchmarks over my test networks, since I'm more interested in small-packet forwarding performance. I'm sure with the right tuning (and probably some of the recent work on TCP BIC etc.) you can saturate a 1000-base-tx link with a single tcp flow, no question on that. But at what expense? you copy data back and forth between different address spaces (kernel/user), you have lots of complex code that deals with retransmissions, protocol demultiplex, ... that is totally unneccessarry for the gnuradio/USRP kind of application. Systems like NetBSD have moved to zero-copy TCP. I am surprised if Linux has not done the same. Perhaps more important is that I do not see any reason, in principle, that you cannot do the same for UDP+RTP. (API adaptations may be necessary.) Dave -- David Young OJC Technologies [EMAIL PROTECTED] Urbana, IL * (217) 278-3933 ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] DBS_RX troubles
I have not seen this message once on mine. Bob Marcus D. Leech wrote: Matt likely isn't back from his trip to Green Bank yet, but I've observed that with the DBS_RX, using dbs_debug, that it complains whenever you go to set a frequency that VCO failed to lock at x appears. Since that particular operation is the only one that seems to get actively tested for a correct result on the MAX2118, I'm wondering if mine is perhaps at least partially DOA, or is this failure to lock to be expected? ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
[Discuss-gnuradio] Gig-E alternatives
Though GigE sounds like a good idea to pursue, has anyone thought about using 2 or more USB 2 interfaces as an alternative? I don't know what the typical controller interfaces can handle, but I see three on my AMD64-based laptop, plus a firewire interface - that's potentially about 1.6 Gb/sec combined, but I only have a 100 Mb/s Ethernet interface. The other thought is that if you are considering putting the peripheral remotely close to an outdoor antenna, perhaps an optical fiber solution would be better - why risk frying your CPU or your body? ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] Gig-E alternatives dual USB2
The other thought is that if you are considering putting the peripheral remotely close to an outdoor antenna, perhaps an optical fiber solution would be better - why risk frying your CPU or your body? Copper GigE is sufficiently cheap and ubiquitous that a DAC/ADC board should use it. But for the people who want to mount the whole thing on a tower and run a nonconducting fiber back to their processing shack, a converter from copper GigE to fiber GigE is small, low power, and only costs a few hundred dollars. Though GigE sounds like a good idea to pursue, has anyone thought about using 2 or more USB 2 interfaces as an alternative? A good hack! I don't know if common multi-USB2 host interfaces can run all the ports at top speed simultaneously. Some look to software like a single interface connected to a hub. I wonder if we could make a Second USB2 daughtercard that would use some of the FPGA's digital I/O pins to drive another USB2 chip? (The protocol over the USB would need to change to add packet numbers, or to otherwise provide for a way to synchronize the two streams; but this is probably a simpler change than making a GigE interface.) John ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
[Discuss-gnuradio] PYTHONPATH issue regarding missing Numeric module during CVS builds
There was earlier some discussion surrounding why Numeric cannot be found during the build tests http://lists.gnu.org/archive/html/discuss-gnuradio/2005-04/msg00135.html The problem is manifested by a console message to the effect ImportError: No module named Numeric I encountered the same problem, and fixed by changing PYTHONPATH=$swigbld:$swigbld/.libs:$swigsrc:$py to PYTHONPATH=$PYTHONPATH:$swigbld:$swigbld/.libs:$swigsrc:$py in the file gr-build/gnuradio-core/src/python/gnuradio/gr/run_tests.in Prior to the for-all-dirs build process, PYTHONPATH must contain an element that points to the Numeric module. For example PYTHONPATH=/home/petrovic/gr/lib/python2.3/site-packages:/home/petrovic/gr/lib/python2.3/site-packages/Numeric:/usr/lib/python2.3/site-packages export PYTHONPATH per KL7LMO's Numeric installation instructions http://www.kd7lmo.net/ground_gnuradio_ota.html Mark ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
[Discuss-gnuradio] gnuradio-core-2.5 make error
I get the following error when I 'make' I am using gcc-3.2.3 and Python2.3. How can I fix it? Thanks in advance. [EMAIL PROTECTED] gnuradio-core-2.5]$ make make all-recursive make[1]: Entering directory `/home/sheikh/gnuradio-core-2.5' Making all in config make[2]: Entering directory `/home/sheikh/gnuradio-core-2.5/config' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/home/sheikh/gnuradio-core-2.5/config' Making all in src make[2]: Entering directory `/home/sheikh/gnuradio-core-2.5/src' Making all in gen_interpolator_taps make[3]: Entering directory `/home/sheikh/gnuradio-core-2.5/src/gen_interpolator_taps' make[3]: Nothing to be done for `all'. make[3]: Leaving directory `/home/sheikh/gnuradio-core-2.5/src/gen_interpolator_taps' Making all in lib make[3]: Entering directory `/home/sheikh/gnuradio-core-2.5/src/lib' Making all in missing make[4]: Entering directory `/home/sheikh/gnuradio-core-2.5/src/lib/missing' if /bin/sh ../../../libtool --mode=compile g++ -DHAVE_CONFIG_H -I. -I. -I../../.. -g -O2 -Wall -Woverloaded-virtual -pthread -MT bug_work_around_8.lo -MD -MP -MF .deps/bug_work_around_8.Tpo -c -o bug_work_around_8.lo bug_work_around_8.cc; \ then mv -f .deps/bug_work_around_8.Tpo .deps/bug_work_around_8.Plo; else rm -f .deps/bug_work_around_8.Tpo; exit 1; fi mkdir .libs g++ -DHAVE_CONFIG_H -I. -I. -I../../.. -g -O2 -Wall -Woverloaded-virtual -pthread -MT bug_work_around_8.lo -MD -MP -MF .deps/bug_work_around_8.Tpo -c bug_work_around_8.cc -fPIC -DPIC -o .libs/bug_work_around_8.o /bin/sh ../../../libtool --mode=link g++ -g -O2 -Wall -Woverloaded-virtual -pthread -o libmissing.la bug_work_around_8.lo ar cru .libs/libmissing.a .libs/bug_work_around_8.o ranlib .libs/libmissing.a creating libmissing.la (cd .libs rm -f libmissing.la ln -s ../libmissing.la libmissing.la) make[4]: Leaving directory `/home/sheikh/gnuradio-core-2.5/src/lib/missing' Making all in runtime make[4]: Entering directory `/home/sheikh/gnuradio-core-2.5/src/lib/runtime' if /bin/sh ../../../libtool --mode=compile g++ -DHAVE_CONFIG_H -I. -I. -I../../.. -I../../../src/lib/runtime -I../../../src/lib/general -I../../../src/lib/filter -I../../../src/lib/io -I../../../src/lib/g72x -I../../../src/lib/atsc -I../../../src/lib/omnithread -I../../../src/lib/general -I../../../src/lib/filter -I/home/sheikh/gr/include -I/home/sheikh/gr/include-g -O2 -Wall -Woverloaded-virtual -pthread -MT gr_block.lo -MD -MP -MF .deps/gr_block.Tpo -c -o gr_block.lo gr_block.cc; \ then mv -f .deps/gr_block.Tpo .deps/gr_block.Plo; else rm -f .deps/gr_block.Tpo; exit 1; fi mkdir .libs g++ -DHAVE_CONFIG_H -I. -I. -I../../.. -I../../../src/lib/runtime -I../../../src/lib/general -I../../../src/lib/filter -I../../../src/lib/io -I../../../src/lib/g72x -I../../../src/lib/atsc -I../../../src/lib/omnithread -I../../../src/lib/general -I../../../src/lib/filter -I/home/sheikh/gr/include -I/home/sheikh/gr/include -g -O2 -Wall -Woverloaded-virtual -pthread -MT gr_block.lo -MD -MP -MF .deps/gr_block.Tpo -c gr_block.cc -fPIC -DPIC -o .libs/gr_block.o if /bin/sh ../../../libtool --mode=compile g++ -DHAVE_CONFIG_H -I. -I. -I../../.. -I../../../src/lib/runtime -I../../../src/lib/general -I../../../src/lib/filter -I../../../src/lib/io -I../../../src/lib/g72x -I../../../src/lib/atsc -I../../../src/lib/omnithread -I../../../src/lib/general -I../../../src/lib/filter -I/home/sheikh/gr/include -I/home/sheikh/gr/include-g -O2 -Wall -Woverloaded-virtual -pthread -MT gr_block_detail.lo -MD -MP -MF .deps/gr_block_detail.Tpo -c -o gr_block_detail.lo gr_block_detail.cc; \ then mv -f .deps/gr_block_detail.Tpo .deps/gr_block_detail.Plo; else rm -f .deps/gr_block_detail.Tpo; exit 1; fi g++ -DHAVE_CONFIG_H -I. -I. -I../../.. -I../../../src/lib/runtime -I../../../src/lib/general -I../../../src/lib/filter -I../../../src/lib/io -I../../../src/lib/g72x -I../../../src/lib/atsc -I../../../src/lib/omnithread -I../../../src/lib/general -I../../../src/lib/filter -I/home/sheikh/gr/include -I/home/sheikh/gr/include -g -O2 -Wall -Woverloaded-virtual -pthread -MT gr_block_detail.lo -MD -MP -MF .deps/gr_block_detail.Tpo -c gr_block_detail.cc -fPIC -DPIC -o .libs/gr_block_detail.o if /bin/sh ../../../libtool --mode=compile g++ -DHAVE_CONFIG_H -I. -I. -I../../.. -I../../../src/lib/runtime -I../../../src/lib/general -I../../../src/lib/filter -I../../../src/lib/io -I../../../src/lib/g72x -I../../../src/lib/atsc -I../../../src/lib/omnithread -I../../../src/lib/general -I../../../src/lib/filter -I/home/sheikh/gr/include -I/home/sheikh/gr/include-g -O2 -Wall -Woverloaded-virtual -pthread -MT gr_buffer.lo -MD -MP -MF .deps/gr_buffer.Tpo -c -o gr_buffer.lo gr_buffer.cc; \ then mv -f .deps/gr_buffer.Tpo .deps/gr_buffer.Plo; else rm -f .deps/gr_buffer.Tpo; exit 1; fi g++ -DHAVE_CONFIG_H -I. -I. -I../../.. -I../../../src/lib/runtime -I../../../src/lib/general -I../../../src/lib/filter -I../../../src/lib/io
