Juha,
I'm worried about truncating too far because I could be close to the
emitters which would cause a large SNR which (as far as I understand) also
means more dynamic range. Losing bits will chop dynamic range :( Now, I
assume that as long as I am on top of the RX gain I can ensure that I don't
get massive SNR signals thus not needing the extra bits. Please correct me
if that's wrong as I'm not 100% sure that's right.
As for the various parallel compression algorithms, I found a list at
http://askubuntu.com/questions/258202/multi-core-compression-tools (about
half way down) of tools. I tried out pbzip2 earlier with `cat
largeFile.sc16 | pzip2 -1 | pv > /dev/null` and found that it was only
running at ~ 25 MBytes/s which is too slow. That number is from my i7
4870HD Macbook Pro. If I run pgzip with `cat largeFile.sc16 | pigz -1 | pv
> /dev/null` I get around 120 MBytes/s. Also, the file was stored in
/dev/shm for best results. I know that my desktop machine which will be
doing the recording can run an sc16 file through at around 300-350 MBytes/s
but that maxes out the entire machine. Do you know of any tweaks that can
be done to increase the speed with a slight trade off in compression?
Marcus,
I should probably have stated the problem a little more clearly. I
would like to be able to record 100+ MHz swaths of busy spectrum. Think
900 MHz in a busy city or LTE, WiFi, 433 MHz, etc. Just trying to build up
a large collection of signals to learn how to demodulate with. I also hope
one day to be able to pull down entire transponders of satellites :D So,
the basic point here is that there will be a lot going on in the spectrum
that I am recording. Oh, and the captures could be several minutes long.
Sometimes I find really interesting signals that look crazy in the
waterfall (fosphor) plot so I want to record large files of those to poke
at later.
I'm really intrigued by your idea of storing the log magnitudes. It
will take me a bit to wrap my head around it, but it sounds really
interesting :D
Thank you both very much!!
-Dave
On Sat, Jul 16, 2016 at 4:29 PM, Marcus Müller <marcus.muel...@ettus.com>
wrote:
> Hello Juha,
>
> idea: if Dave's distribution of amplitudes was a little more benign than
> the Radar near/far problem, and he would favor full resolution when the
> signal is weak, but could live with a bit of degradation due to
> quantization when the signal is strong, what about storing a logarithm of
> the I and Q magnitudes instead of their linear value? That way, he could
> get the same weak-signal-resolution with less bits, and only suffer
> inaccuracies due to quantization for situations where signal is strong,
> anyway.
> Best regards,
> Marcus
>
>
> On 16.07.2016 21:57, Juha Vierinen wrote:
>
> Can you reduce the number of bits that you are using?
>
> With radar signals, the receiver noise most of the time excites only about
> 8 bits out of 16. Ground clutter or meteor echoes excite nearly all of the
> bits occasionally, so I can't just truncate to 8 bits. In this case, bzip2
> actually does a pretty good job of getting rid of the 8/16 most significant
> bits that are zero most of the time. Thus, I get a compression ratio close
> to 50% when using sc16. pbzip2 is a good tool for doing parallel
> compression on files.
>
> juha
>
> On Sat, Jul 16, 2016 at 3:59 AM, Dave NotTelling <dmp250...@gmail.com>
> wrote:
>
>> Anyone have experience with streaming 100+ MSPS of 16 bit IQ data through
>> a compression tool and then to disk? I'd like to be able to take really
>> wide band snaps for several minutes. Currently that would take up 16 * 2 *
>> SAMP_RATE bits per second. So, for 200 MSPS that would end up being 800
>> MBytes/s. That rate eats up a hard drive pretty quickly. Running it
>> through gzip by way of pigz was my first thought, but even on an 8 core
>> Intel machine pigz just can't keep up. I can sustain maybe 300 MBytes/s
>> but that's it. And I know it's not a hard disk limitation as the M.2 drive
>> I am using can easily sustain 800 MBytes/s of uncompressed data.
>>
>> Thoughts?
>>
>> Thank you!
>>
>>
>> -Dave
>>
>> _______________________________________________
>> Discuss-gnuradio mailing list
>> Discuss-gnuradio@gnu.org
>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>>
>>
>
>
> _______________________________________________
> Discuss-gnuradio mailing
> listDiscuss-gnuradio@gnu.orghttps://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
>
>
> _______________________________________________
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
>
_______________________________________________
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
