Re: [Discuss-gnuradio] sparc -- x86 data exchange
Fedora Extras now has liboil, and several media player apps now depend on it, so I think finding it installed or at least packaged on a given system is very likely. It doesn't seem to be in pkgsrc yet, but I just built it on NetBSD/i386 and it passed make check. So, I should be able to make a package in only a few minutes, and that should also be true for FreeBSD and OpenBSD ports. It seems to only have optimized support for i386 and powerpc, but I suppose that's pretty much what people are using. It would be unfortunate if performance using it goes down relative to current levels on other systems (e.g. sparc64). -- Greg Troxel [EMAIL PROTECTED] ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] sparc -- x86 data exchange
On Tuesday 20 December 2005 01:04, Eric Blossom wrote: I disagree. But we can have the best of both worlds. When logging lots of binary diagnostic data to files, the last thing I want is for it to take longer. The time to swap bytes or words in the output stream will be insignificant by three or four orders of magnitude compared to the disk's write performance. Yes, three or four orders of magnitude; byte-swapping is a single instruction, typically. Disk write performance is the killer, and one or even a hundred instructions in the pipe aren't going to make a dent in the slow speed of disk. And this is regardless of the caching used by the disks: I found by experimentation with a dual Xeon 3 GHz box, with a PCI Express caching U320 SCSI RAID controller, that the system is I/O (disk and USB, which have to share the same processor bus, unfortunately) bound, even when performing work on the data in the pipe. There's nothing stopping anyone from writing: gr_file_sink_big_endian gr_file_sink_little_endian gr_file_source_big_endian gr_file_source_little_endian Certainly. But if I save a file on one arch it should be able to be played back on another without me having to write anything. One of the keys of GNUradio is the fact that the scripts themselves are arch agnostic and portable across platforms; requiring different I/O blocks per platform breaks that. I mean, I could easily see something like an Xscale or MIPS embedded processor running a stripped down kernel and GNUradio for a portable software radio device. A USB2 capable pocket PC or similar could actually have something like the SSRP or a half-USRP embedded, and with a color LCD could become a portable spec-an/oscope/etc. So I think the byte-ordering thing is important, and I further think that regardless of platform that the input and output files always have the same byte order regardless of platform. Did a lot of it, long ago. (Wrote a complete WYSIWYG editor for Unix in the days when a 25 MIPS workstation was a very cool thing to have). Not the kind of work I'm particularly interested in doing these days. Maybe it was the 90 hr work weeks ;) The toolkits today are much better, but I agree with you. My frontpanels on equipment I have made aren't works of art; the circuits inside might be, but the frontpanels certainly are not. Sounds like a good application to emulate. Volunteers? FWIW, I can provide screenshots of SpectraVue in operation. Given time I might be able to emulate it; but I'd like to go past it, and that's where my abilities in graphics design fall apart. And I'm more of a backend person, too, and much more interested in the radio astronomy aspects, which aren't even necessarily GUI in nature. I did have a brainstorm early this morning, in that a GUI block-building app would be way cool, something like those Lego building apps where you can drag and drop blocks around. Doing that for GNUradio would be seriously cool. I don't have the foggiest idea at this point how to implement such a thing, but it would still be cool. An IDE for GNUradio, with the Boa Constructor GUI builder, and a backend flowgraph builder that would let you wire GUI elements to block signals, or something like. What one would do is be able to choose from a pallet of blocks, wire them together, and have the app generate the code to plug the GUI into. What I envision is something like what Xcircuit does for schematic capture and SPICE analysis (Xcircuit can generate the netlists to feed spice)... hmmm, Xcircuit is scriptable in Python...perhaps Xcircuit is already the frontend, but needs glue to tell it how to generate a flow graph rather than a spice netlist. Take the circuit emulation of GNUradio to the next level, and design using circuit-level tools. Of course, GNUradio is much higher level than spice, but I think the analogy holds pretty well. manufacturers. Frontpanel and functionality design are non-trivial. But with its solid foundation, GNUradio has the potential to make the guts less of an issue, and the frontpanel/functionality can become the prime goal. Nicely put. Thank you. What I see GNUradio's niche, as the code currently stands, is a kit of lots of parts that you can wire together. You need to learn how to solder (Python), you need to learn what each component does, and you need to a have a grasp of what you want the result to do. Building devices from parts is non-trivial; image having to build those parts. GNUradio is like a kit of IC's ready made that you need to wire together and solder (and test, and package, and...). It makes software radio easier to do, but it is still a kit. -- Lamar Owen Director of Information Technology Pisgah Astronomical Research Institute 1 PARI Drive Rosman, NC 28772 (828)862-5554 www.pari.edu ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org
Re: [Discuss-gnuradio] sparc -- x86 data exchange
[back onlist; my fault for not reply-all]
On Tuesday 20 December 2005 14:41, Eric Blossom wrote:
On Tue, Dec 20, 2005 at 11:37:02AM -0500, Lamar Owen wrote:
Certainly. But if I save a file on one arch it should be able to be
played back on another without me having to write anything.
I understand the inter-machine portability issue. I prefer native
because I often manipulate the files outside of GNU Radio. Some of
the tools I use can read and write in either format, but it's also
nice to be able to mmap the files in and use them directly. Think
about files in the GB plus category..
I'm looking at a 116GB pulsar data capture file on that dual Xeon machine
right now. 2 hours continuous capture, 25 USB underruns.
Perhaps we add an additional endian argument to
gr_file_{sink,source} where the default is read from a configuration
file. You could set the default on your systems to big, I'd set
mine to native and we'd both get what we'd want, and the scripts
remain arch agnostic.
Hmm, perhaps.
We could also use reader makes right, but that would require that we
tag the files with a header of some kind or use a file naming
convention. Not having a header is very convenient, but again,
there's room for discussion.
Perhaps WAV format could be investigated, which would open things up
dramatically. Of course, there's overhead, too. WAV is standard, and lots
of tools and libraries are available for WAV. In the absence of WAV, FITS
format is the next best fit (pun intended).
The other problem with all of this, is that the file sinks and source
don't actually *know* the format of the data.
Without knowing the structure, blind swapping won't work.
Yeah, it either has to always be the same or metadata has to be added. Either
WAV or FITS have lots of libraries available to do that.
Right now we have very few complete apps. Emulating SpectraVue would
be a good start. Making it better can be left for later.
I'll grab some screens, perhaps Thursday or Friday (very very busy today and
tomorrow).
What one would do is be able to choose from a pallet of blocks, wire
them together, and have the app generate the code to plug the GUI into.
Tad Dreier at UCLA is looking at gluing GNU Radio into Ptolemy II.
This would give us something along these lines.
http://ptolemy.eecs.berkeley.edu
Tad, now would be a good time to de-cloak ;)
Cool. Java? Hmmm, seems heavy-weight, but I'll withhold criticism (after
all, our Smiley remote controlled radio telescope uses Java).
GNUradio
is like a kit of IC's ready made that you need to wire together and
solder (and test, and package, and...). It makes software radio easier to
do, but it is still a kit.
Now we need some directions and a front panel for those radio kits ;)
Precisely.
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC 28772
(828)862-5554
www.pari.edu
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] sparc -- x86 data exchange
At 05:12 PM 12/16/2005 -0800, you wrote: ambulance frequencies, or TiVo the ham bands, or avoid DRM on John - I already have a (to me) nice little LW/MW/SW TiVo already packaged and ready to go. I use it daily and have added everything I can think of: AM, Upper/Lower Sideband, Independant Sideband, bandswitching, 30 audio filters, FFT display, rewind/fast forward when playing back a recording, command line switches to set decimation, center frequency and file to record to or play back from (if any), slider adjustable rf gain (pga), makes use of a Powermate knob for controls, etc. All it needs is a good antenna and about 40-50db gain, a basic RX board and existing software and you have something IMHO better than a TenTec RX-320. The only thing missing is marketing and promotion, and probably some customer feedback and redesign. I use it a lot to TiVo while listening at night, then do some more listening the next day at lunch. A 2.x version of HDTV would be neat - I could drive up to the antenna, record about an hour or two of 6-8Mhz wide signal to my portable system, process it overnight and have an hour or two long program ready the next day ;) Someone could make a hobby out of collecting DVD's of football games and movies and would be ready to deal with that nasty broadcast flag when it appears. --Chuck kb4new ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] sparc -- x86 data exchange
At 08:00 PM 12/15/2005 -0500, Dave Dodge spoke and all was made clear: sparc: 11 22 33 44 x86: 44 33 22 11 OIC. Ok, switching to 2-byte short data and using dd conv=swab, a dial tone file created on a sparc plays fine on the intel box now. Thanks very much. Ideally the endian conversions would be done in processing blocks. --Chuck ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] sparc -- x86 data exchange
On Fri, Dec 16, 2005 at 03:04:01PM -0800, Matt Ettus wrote: [..] This would actually be a good place to make use of liboil, the library of optimized inner loops. They have code for all sorts of conversions, and they pick up MMX and SSE-type optimizations automatically. Indeed. liboil is really promising. I was waiting for GNU Radio 2.6 to be released, and provided you and Eric agree, IMHO it would be a smart move to outsource the SIMD speedups to liboil for the next GNU Radio release. It's way easier to develop speedups in liboil, introduce new archs, have them improved by people other than the GNU Radio gang (IOW share efforts and benefits), etc. Right now, I'm creating the missing liboil API entries along with reference implementations. Reference implementations are plain C code which implements the algorithm in a clear, readable, high- precision and always right way. Most of the time, reference implementations are horribly slow. But it's not a problem. Based on the reference prototype, you may write as many optimized implementations as you want. The liboil framework makes it easy to automate testing against the reference implementation and corner cases (spill, alignment, ..). Always wondered whether SSE or 3DNow! is faster on some new AMD chip? Don't care about that, liboil benchmarks at run time your breed(1) of optimized implementations and selects the one which has the best rating. (1) not *genetically* engineered so far, sorry Eric ;-) Expect liboil to gain more momentum, and pass the word among the other SDR projects you may be involved in. One thing I will need quickly is the list of inner loops in the need of optimization (OProfile reports anyone?). This is to begin with creating the API entries and the reference implementations. Then, making it familiar to some ears, liboil is The library that just keeps getting optimized better. See http://liboil.freedesktop.org/wiki/FrontPage Definitely worth mentioning! -- Stephane ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] sparc -- x86 data exchange
Three opinions... * GNU Radio should be processing data in the local machine's native byte order and data format (e.g. IEEE float). You should have to explicitly tell it to do something about byte order (e.g. convert samples to little-endian). * Any conversions we offer should say nothing about machine types (x86), rather should be specific about byte orders. E.g. we should not offer convert longs from x86 to sparc, we should offer output big-endian longs and input big-endian longs. These conversions would be implemented on all machine types, but of course on some of them there's no work to do. An extra name for big-endian should be in network byte order, for people who are squirting partially-processed data over the net to another signal processing program on an arbitrary machine. (My opinions on byte order come from having co-designed and maintained the BFD library that the GNU binutils use to read and write object files, which require pervasive and specific attention to byte order. We changed our byte-order API several times until we got it right and easy to use.) * I'm hesitant to make GNU Radio depend on Yet Another random library package like liboil. Building it is already an exercise in frustration, as is trying to package it for a distribution. Can't we skip some premature optimization and get back to making the program do real things that real people want to do? I don't see the obvious GUI-operable scanners, recorders, radar screens, etc, that our capabilities are supposed to make it easy to create. We're how many years into this package?, and still re-re-rewriting the guts. Let's rather make it something that tens of thousands of people would use to record multiple FM stations, or scan and log police and ambulance frequencies, or TiVo the ham bands, or avoid DRM on over-the-air TV transmissions. Even our FM decoding is inferior to what cheap transistor radios do. Six months' focus on polish and applications would make a world of difference. Just my 2c... John ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] sparc -- x86 data exchange
On Thu, Dec 15, 2005 at 03:29:38PM -0500, cswiger wrote:
Gang - any hints on what happens to float data when transfered
from a sparc machine to an x86? I understand byte-swap but this
looks like word-swap.
The problem is that the default settings for od are almost never
what you actually want, and are making things confusing.
What od is doing is treating the file as a sequence of 16-bit
integers in native byte order. On a little-endian machine this means
that od is _not_ showing you the order that the bytes actually appear
in the file. Since the values you're really interested in are 32
bits, it's also not grabbing the right number of bytes for display.
Assume the 32-bit value in the application's memory is 0x11223344.
When you write this to a file in native byte order, you get the byte
sequence:
sparc: 11 22 33 44
x86: 44 33 22 11
If you od each file on its own system, you get:
sparc on sparc: 000 1122 3344
x86 on x86: 000 3344 1122
This has the appearance of the word swapping you were seeing. Now,
if you move the sparc file over to x86 and od it, you get:
sparc on x86: 000 2211 4433
Which looks like byte swapping. If you then use dd to byte-swap the
contents of the file, you get the file:
sparc swapped: 22 11 44 33
When run through od on x86:
sparc swapped on x86: 000 1122 3344
That matches the od output from sparc on sparc, but od isn't looking
at the right thing so it's still not right. If x86 reads a 32-bit
value from these files, neither of the sparc files will work:
x86 reading from x86: 0x11223344 -- correct
x86 reading from sparc: 0x44332211
x86 reading from sparc swapped: 0x33441122
So assuming the application is actually reading and writing this data
as 32-bit native words, what you need to do is reverse the order of
each _four_ bytes of the file when moving between sparc and x86.
Here's a bit of perl code to try. I'm sure there's much better tools
for doing this same thing, or a perl one-liner that works faster, but
this should at least get you started:
#!/usr/bin/perl
while(1){
my $a = getc || exit;
my $b = getc || exit;
my $c = getc || exit;
my $d = getc || exit;
print $d,$c,$b,$a;
}
When using od with these files, what you really want to do is
either:
od --format=x1
which will show you the actual byte ordering in the file.
od --format=x4
which will read 32-bit words from the file in the machine's byte
order and show you what values the application is seeing. When
you've got the files convered properly, you should see the same
result from this on both sparc and x86.
For data files intended to be portable, or when transmitting over the
network, the best solution is to pick a fixed byte ordering for the
file and do your byte reordering during I/O. In the TCP/IP networking
world, big-endian (such as used by Sparc) is commonly used in network
protocols and is sometimes called network order; there's little
functions like htonl and ntohs that can convert values between
host and network byte order in memory.
-Dave Dodge
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
