Hi Dirk: In the IQ data file you provided I can seem to find any pulses of the nominal 10 msec length, no matter how I filter and rotate the spectrum. There is a lot of EMI, which looks like the intermodulation products of a continuously on guy who is drifting/chirping down in frequency.
So could you please confirm or clarify the following: 1. The format of the binary data file. I am assuming 32 bit float I and 32 bit float Q pairs, as that is what the GNURadio file sink would normally create. 2. The sample rate. I am assuming 1 Msps. 3. The center freq. I am assuming 150.22 MHz. 4. The pulse duration. I am assuming 10 milliseconds. 5. At what frequency offset, from the center frequency, should the pulse be at? I'm assuming somewhere withing +/- 5 kHz of the center spike, but there are at least two EMI spikes in that range. Thanks. -Andy On Wed, Mar 15, 2017 at 5:23 AM, Dirk Gorissen <dgoris...@gmail.com> wrote: > Hi Andy, Marcus, > > Thanks very much for taking the time to think about this. > > Just to answer your questions: > >>Dirk's problem was the low SNR in his recording, so he'd like to take >>the shape of his pulse into consideration, but he doesn't know the >>spectral position of the same – Dirk, can you confirm? > > Yes, Im dealing with a very weak signal against quite a bit of > background noise so the more prior knowledge I can leverage the > better. > >>Dirk, thinking about this, the relation of the spectral uncertainty to >>the bandwidth of the pulse would be interesting – can you refresh our >>memory? I think the pulse was 10ms long (so, pretty long) but I forgot >>how it looked like. > > Yes, 10ms long (this is pretty exact), occurring every 1.5 seconds, in > the 150MHz range. My input stream is coming from an SDR. > As an aside I actually have multiple transmitters each pulsing at > slightly different frequencies (e.g., 150.10, 150.22, ...) but Im > happy to treat them independently so you can ignore that and assume > there is just one. > > You can see pictures of the pulse (taken a while back, for a specific > tx frequency) here: https://goo.gl/photos/Y3Ea6kJo1ewrcDYM8 > > Note though that this is taken with the transmitter very close to the > antenna so the signal is unrealistically strong. So its just to > illustrate. > > I also quickly captured a file with raw IQ samples (File sink) in case > anybody is interested. It starts with the transmitter very close to > antenna, moves progressively further until out of range and then back. > Its only about a minute or two tops but at 1msps it ended > up as 813 MB though. > > https://drive.google.com/drive/folders/0B5dKo9igl8W4dWpiMmlEYWx3b0k?usp=sharing > >>What you really care about is >>that there's a *tone* for about 8ms < t < 12ms where there is none else >>(to rule out detection of spurs and other interference), is that right? > > Yes. The "tone" happens every 1.5 seconds and I want to catch as many > of those occurrences as possible as the receiver moves through space. > So for every 1.5 seconds I need to make a decision: was there one, or > not. > > Note that the receiver is moving. So perhaps initially I may be well > out of range of the signal and get nothing. But as I move closer as > some point I will start picking up those pulses. The earlier and more > reliably I can pick them up the better. > > I actually did spend some time looking at PLLs & the gnu radio block > at some point. However I readily admit to being a software person not > a DSP/Radio person. I have the day job to deal with today but I will > have a study of the PLL block given Andy's tips and report back. > > Cheers > Dirk > > > > > > On 14 March 2017 at 16:37, Andy Walls <a...@silverblocksystems.net> wrote: >> On Tue, 2017-03-14 at 12:00 -0400, discuss-gnuradio-requ...@gnu.org >> wrote: >>> Date: Tue, 14 Mar 2017 15:49:44 +0100 >>> From: Marcus M?ller <marcus.muel...@ettus.com> >>> To: discuss-gnuradio@gnu.org >>> Subject: Re: [Discuss-gnuradio] fast parallel filtering >> >>> Hi Andy, >>> >>> Dirk's problem was the low SNR in his recording, so he'd like to take >>> the shape of his pulse into consideration, but he doesn't know the >>> spectral position of the same ? Dirk, can you confirm? >> >> Ah, OK. >> >> >>> Dirk, thinking about this, the relation of the spectral uncertainty to >>> the bandwidth of the pulse would be interesting ? can you refresh our >>> memory? I think the pulse was 10ms long (so, pretty long) but I forgot >>> how it looked like. >>> >>> Having slept about this a couple of times: What you really care about >>> is >>> that there's a *tone* for about 8ms < t < 12ms where there is none >>> else >>> (to rule out detection of spurs and other interference), is that >>> right? >>> >>> Maybe we've been focussing too much on filter-based detection. What if >>> we just *use* that feature of the signal being periodic for a duration >>> of t, and not else? A PLL would be able to "lock" on the tone (much >>> like >>> an FM Radio with a PLL will lock on the station's carrier), and if we >>> observe the phase error being limited for t, we can derive there was a >>> pulse. >>> >>> Andy, does that sound like a feasible approach? >> >> Yes. With the added benefit of GNURadio's carrier tracking PLL actually >> performing part of the correlation operation for you, if it works out. >> >> So use the carrier_tracking_pll block. If it locks onto a good signal, >> it will mix it down to DC. Then all you need is an averaging filter, >> like the single pole IIR filter block or the moving average filter >> block, operating on the real part of that output. That will act as a >> lock detector, and, I think, also completes a correlation operation, if >> you use a non-normalized moving average filter (since the carrier >> tracking PLL block gives you a point by point complex multiply with the >> conjugate of the complex carrier tone). >> >> You'll want to set the PLL allowed min and max frequency bounds >> properly; be careful since the input arguments have tricky units. >> >> You'll also want the loop bandwidth set pretty wide, since you want to >> lock-in rapidly. >> >> Also, if I'm thinking about this properly: >> To get faster lock in, you may want your frequency range of interest to >> be somewhere in between Fs/4 and Fs/2; and not near DC. If you're >> within the lock-in range of the PLL, you'll lock within 1 cycle. If >> you're in the pull-in range of the PLL, it can take many cycles to get >> into lock. >> >> Regards, >> Andy >> >>> Cheers, >>> >>> Marcus >>> >>> >>> On 14.03.2017 14:18, Andy Walls wrote: >>> > If there is no information on the signal, why not just do a straight >>> > energy detector: >>> > >>> > source -> complex bandpass filter -> complex to mag -> burst/energy >>> detector -> QT Time sink (triggering on start of burst tag) >>> > >>> > The complex bandpass filter just has to catch all the frequencies of >>> > interest. (A complex bandpass filter does not have a symmetrical >>> image >>> > around DC.) >>> > >>> > The burst/energy detector has to detect some minimum number of time >>> > domain samples contiguously above some noise/signal threshold that >>> you >>> > set, and tag the pulse. It also has to detect when the time domain >>> > samples fall below the threshold. The burst/energy detector can >>> work >>> > with a preset noise/signal threshold, or you could periodically >>> > re-estimate that threshold. >>> > >>> > GNURadio does not have a stock block that does burst/energy >>> detection, >>> > so you'll have to find one on the 'Net or write one yourself. >>> > >>> > -Andy >>> > >>> > >>> > Dirk Gorissen wrote: >>> >> Hi Martin, >>> >> >>> >> The aim is to check for the existence of a 10ms pulse in the >>> incoming >>> >> radio signal (from an sdr). The thing is I only know the frequency >>> of >>> >> the pulse to within ~1khz. >>> >> >>> >> So a single filter in my case is: generate a synthetic pulse >>> (complex >>> >> sin wave) of the same length and with a frequency f. Then take the >>> >> reverse of the complex conjugate of this pulse to give me the taps >>> for >>> >> a FIR filter. >>> >> >>> >> Repeat the above for each frequency I want to check. E.g., 10 >>> filters >>> >> each 100Hz apart. >>> >> Then I just take the magnitude of each filter output and push >>> through >>> >> a Max to get my final correlations. >>> >> >>> >> I can come up with an algorithm that tries to be clever and with >>> some >>> >> accounting tries to find what frequency matters most but I wouldnt >>> be >>> >> surprised if there is some theory you can use to do this more >>> >> efficiently or even in a single shot. But this is where Im unsure. >>> >> >>> >> Cheers >>> >> Dirk >>> >> >>> >> >>> >> On 14 March 2017 at 01:09, Martin Braun <address@hidden> wrote: >>> >>> How related are those filters? Is this a candidate for polyphase >>> DSP? >>> >>> >>> >>> -- M >>> >>> >>> >>> On 03/11/2017 02:01 PM, Dirk Gorissen wrote: >>> >>>> Hi Marcus, >>> >>>> >>> >>>> Sorry, I should have clarified. You may recall an earlier thread >>> from >>> >>>> mine where Im looking to pick out a short pulse from background >>> noise >>> >>>> but I dont know the exact frequency of the pulse. Thought I >>> would >>> >>>> start a new thread with a clear, specific question. >>> >>>> >>> >>>> There is an uncertainty of +/- 1khz. So I can define multiple >>> filters, >>> >>>> correlating for different pulse frequencies across the 1khz >>> range. I >>> >>>> can implement this with different filters running in parallel >>> but I >>> >>>> was looking for a more flexible / efficient way. >>> >>>> >>> >>>> If you think this is out of scope for this list no problem at >>> all, >>> >>>> just let me know. >>> >>>> >>> >>>> Cheers >>> >>>> Dirk >>> >>>> >>> >>>>> On 11 March 2017 at 20:02, Marcus M?ller <address@hidden> wrote: >>> >>>>>> Hi Dirk, >>> >>>>>> >>> >>>>>> this is more of a general DSP question than a GNU Radio >>> question: >>> >>>>>> >>> >>>>>> How do these filters relate to each other? >>> >>>>>> >>> >>>>>> My gut feeling is that this gets a lot easier as soon as you >>> tell us why >>> >>>>>> you're doing this, i.e. for what purpose :) >>> >>>>>> >>> >>>>>> Best regards, >>> >>>>>> Marcus >>> >>>>>> >>> >>>>>> On 11.03.2017 19:28, Dirk Gorissen wrote: >>> >>>>>>> Hello all, >>> >>>>>>> >>> >>>>>>> Given a stream of samples I would like to apply n slightly >>> different >>> >>>>>>> filters to it with n being able to be chosen at runtime. Then >>> combine >>> >>>>>>> the results back to a single stream. >>> >>>>>>> >>> >>>>>>> As a test I built a flowgraph with the following chains in >>> parallel for n >>> >>>>>>> = 6 >>> >>>>>>> >>> >>>>>>> | -> decimating fir filter 1 -> complex to >>> mag -> | >>> >>>>>>> stream -> | -> decimating fir filter 2 -> complex to mag -> >>> | -> Max >>> >>>>>>> -> ... >>> >>>>>>> | .... >>> >>>>>>> | >>> >>>>>>> | -> decimating fir filter n -> complex to >>> mag -> | >>> >>>>>>> >>> >>>>>>> So the same stream is sent to each chain (decimation is 1) and >>> the >>> >>>>>>> output of each chain is pushed through a big Max block with 6 >>> inputs. >>> >>>>>>> >>> >>>>>>> This works but not particularly elegant and a bit annoying to >>> change >>> >>>>>>> if I suddenly decide I want to change n. In particular it also >>> does >>> >>>>>>> not seem computationally efficient. >>> >>>>>>> >>> >>>>>>> What I would like is to replace the above by a single block >>> that >>> >>>>>>> >>> >>>>>>> - replicates the input n times >>> >>>>>>> - applies each filter on each replica >>> >>>>>>> - combines the output again to a single stream >>> >>>>>>> - have a tunable n parameter >>> >>>>>>> - is fast >>> >>>>>>> >>> >>>>>>> I did this with an Embedded python block doing essentially >>> this: >>> >>>>>>> >>> >>>>>>> for i in range(n): >>> >>>>>>> out[i] = scipy.signal.lfilter(taps[i], 1, input) >>> >>>>>>> >>> >>>>>>> This is using exactly the same taps as in the chain case. This >>> works >>> >>>>>>> but the output is different and worse than what I get with the >>> >>>>>>> separate chains. As a test, instead of lfilter I tried: >>> >>>>>>> >>> >>>>>>> >>> gnuradio.filter.fir_filter_ccc(1,taps[i]).work(input[0],output) >>> >>>>>>> >>> >>>>>>> Thinking perhaps that is a closer replica. But couldnt get it >>> to work.. >>> >>>>>>> >>> >>>>>>> I suspect there should be an easy / natural way of doing this >>> in >>> >>>>>>> gnuradio. Looked at the filter bank / channelliser blocks but >>> failed >>> >>>>>>> to get anywhere. >>> >>>>>>> >>> >>>>>>> So what is the best way forward to do this? >>> >>>>>>> >>> >>>>>>> Many thanks >>> >>>>>>> Dirk >>> >>>>>>> >>> >>>>>>> _______________________________________________ >>> >>>>>>> Discuss-gnuradio mailing list >>> >>>>>>> address@hidden >>> >>>>>>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>> >>>>>> >>> >>>>>> _______________________________________________ >>> >>>>>> Discuss-gnuradio mailing list >>> >>>>>> address@hidden >>> >>>>>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>> >>>>> >>> >>>>> >>> >>>>> -- >>> >>>>> _________________________________________ >>> >>>>> Dr. Dirk Gorissen >>> >>>>> Mob: +44-7763-806-809 >>> >>>>> Web: dirkgorissen.com >>> >>>>> Skype: dirk.gorissen >>> >>>>> Twitter : twitter.com/dirkgor >>> >>>>> LinkedIn: linkedin.com/in/dirkgorissen >>> >>>> >>> >>>> >>> >>> >>> >>> _______________________________________________ >>> >>> Discuss-gnuradio mailing list >>> >>> address@hidden >>> >>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>> >> >>> >> >>> >> -- >>> >> _________________________________________ >>> >> Dr. Dirk Gorissen >>> >> Mob: +44-7763-806-809 >>> >> Web: dirkgorissen.com >>> >> Skype: dirk.gorissen >>> >> Twitter : twitter.com/dirkgor >>> >> LinkedIn: linkedin.com/in/dirkgorissen >>> > >>> > >>> > _______________________________________________ >>> > Discuss-gnuradio mailing list >>> > Discuss-gnuradio@gnu.org >>> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >> >> > > > > -- > _________________________________________ > Dr. Dirk Gorissen > Mob: +44-7763-806-809 > Web: dirkgorissen.com > Skype: dirk.gorissen > Twitter : twitter.com/dirkgor > LinkedIn: linkedin.com/in/dirkgorissen _______________________________________________ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio