Marcus,
Thanks for the idea anyway :) It sounded like a really neat outside
the box idea.
I ran some quick tests with FLAC and found that it does a really good job
compressing a constant filtered gaussian noise output. I just created a
flow graph with:
fast noise source [float] -> low pass filter [float] -> float to short
-> file sink
The filter was just there so that the spectrum wasn't completely flat.
Running the signal through FLAC with little endian, 16 bit, single channel
resulted in a 2GB file ending up at ~ 400 MB. But, that was just a
constant signal. I need to throw some 900 MHz captures through to see how
that works.
I also found someone who modified FLAC to be multithreaded [1] as well as a
CUDA implementation (windows only) [2]
[1] https://www.akalin.com/parallelizing-flac-encoding
[2] http://www.cuetools.net/wiki/FLACCL
On Sun, Jul 17, 2016 at 7:49 AM, Marcus Müller <marcus.muel...@ettus.com>
wrote:
> Well, having wasted a couple of hours of sleep time on this, maybe you
> shouldn't do the logarithmic storage... at its core its the same idea as
> storing floating point, but with a fixed mantissa.
>
> The mathematical effects of the "rounding to the nearest exponential step"
> on superimposed sinusoids of different amplitude are … funky.
>
> Best regards,
>
> Marcus
>
> On 16.07.2016 21:57, Juha Vierinen wrote:
>
> Can you reduce the number of bits that you are using?
>
> With radar signals, the receiver noise most of the time excites only about
> 8 bits out of 16. Ground clutter or meteor echoes excite nearly all of the
> bits occasionally, so I can't just truncate to 8 bits. In this case, bzip2
> actually does a pretty good job of getting rid of the 8/16 most significant
> bits that are zero most of the time. Thus, I get a compression ratio close
> to 50% when using sc16. pbzip2 is a good tool for doing parallel
> compression on files.
>
> juha
>
> On Sat, Jul 16, 2016 at 3:59 AM, Dave NotTelling <dmp250...@gmail.com>
> wrote:
>
>> Anyone have experience with streaming 100+ MSPS of 16 bit IQ data through
>> a compression tool and then to disk? I'd like to be able to take really
>> wide band snaps for several minutes. Currently that would take up 16 * 2 *
>> SAMP_RATE bits per second. So, for 200 MSPS that would end up being 800
>> MBytes/s. That rate eats up a hard drive pretty quickly. Running it
>> through gzip by way of pigz was my first thought, but even on an 8 core
>> Intel machine pigz just can't keep up. I can sustain maybe 300 MBytes/s
>> but that's it. And I know it's not a hard disk limitation as the M.2 drive
>> I am using can easily sustain 800 MBytes/s of uncompressed data.
>>
>> Thoughts?
>>
>> Thank you!
>>
>>
>> -Dave
>>
>> _______________________________________________
>> Discuss-gnuradio mailing list
>> Discuss-gnuradio@gnu.org
>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>>
>>
>
>
> _______________________________________________
> Discuss-gnuradio mailing
> listDiscuss-gnuradio@gnu.orghttps://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
>
>
> _______________________________________________
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
>
_______________________________________________
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
