Re: [Discuss-gnuradio] Trouble decimating after averaging 4 vectors? Help?

[Jeff Long]

You are already telling GR that you are decimating (nave in decim_block 
init). BTW, you are setting self.nave to 0 in init, instead of using the 
value passed in.


GR should always give you a multiple of 4 vectors (I think) since you've 
specified a decim of 4.


On 03/11/2018 04:03 PM, Glen I Langston wrote:

Thanks Jeff,

I’ll give that a try.

I think I’m also missing something that tells gnu radio what the expected 
decimation is (?),
beyond the decimate = 4 parameter(?).

Also concerning your point, then if I have a non-multiple of 4 vectors I guess 
I need
to keep the sum and count, but I guess that is already a part of the code.

Best regards

Glen





On Mar 11, 2018, at 3:54 PM, Jeff Long  wrote:

in[0] is not the first vector, it is all of the available input for port 0. So, 
you feed in 4 vectors in your test program, and the shape of in[0] is (4,1024). 
You're missing a loop that iterates over the input vectors.

On 03/11/2018 02:06 PM, Glen I Langston wrote:

Hello Gnuradio experts,
I'm following the gnuradio tutorial for creating an out of tree python block.
My intention is fairly simple.  To have a input stream of 4 vectors
and average these 4 vectors, outputting a single average vector
after the 4th vector arrives.  Right now my vectors are 1024 channel
total power spectra.
My code is attached, along with the quality control code.
I've been at this for a while, but I must have the wrong model
in my head on how gnuradio holds these samples.
I'm not getting the proper size matches.   I can not find any
decimation help, despite hours of googling...
Suggestions?
THanks
Glen
My intention is to make this 4 vector average with decimate a
part of the .grc environment, but the problems are easier to diagnose,
as the QA code is stands alone.   Here are the messages I’m getting:
ESMU:vdecimate glangsto$ python qa_vdecimate.py
handler caught exception: could not broadcast input array from shape (4,1024) 
into shape (1,1024)
Traceback (most recent call last):
   File 
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py",
 line 55, in eval
 try: self._callback()
   File 
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py",
 line 160, in __gr_block_handle
 ) for i in self.__out_indexes],
   File "/Users/glangsto/Desktop/Research/vdecimate/vdecimate.py", line 58, in 
work
 out[:] = self.sum/float(self.count)
ValueError: could not broadcast input array from shape (4,1024) into shape 
(1,1024)
thread[thread-per-block[1]: ]: SWIG director method error. 
Error detected when calling 'feval_ll.eval'
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio




___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Trouble decimating after averaging 4 vectors? Help?

[Marcus D. Leech]

On 03/11/2018 03:55 PM, Glen I Langston wrote:

Hi Marcus,

Yes, the suggested combination of IIR filter and the keen one in N Block
would potentially do the job.   However as you pointed out, my end
goal is to have a median of 4 vectors block.  Also I was thinking that
a direct average would be numerically less expensive than the combination
of two different blocks.
For straight averaging/integration I have NEVER had performance issues 
with the

  "IIR followed by keep-one-in-N" idiom.

The number of ops-per-sample for a IIR filter is MUCH less than for a 
moving average of any significant
  window size.   My gut feeling is that doing even a 4-point median 
filter, as you propose, will be somewhat expensive,
  since you need to do some kind of sorting or iterating over the 
window to find and discard the outliers.  For a small window
  size I agree that computationally a single-pole IIR and a 
small-window median would have similar performance, although
  there are these SIMD enhancements that might be awkward to organize 
for a median filter







Also, I’ve been trying for
a few weeks to get an out of tree module to work, so am stubbornly
hoping that I might figure it out.  (Actually I’m hoping someone might
tell me how to figure it out.)

Your suggestion of returning 1 instead of vector size might solve this.
I’ll give it a try.

Best regards

Glen


On Mar 11, 2018, at 3:23 PM, Marcus D. Leech  wrote:

On 03/11/2018 02:06 PM, Glen I Langston wrote:

Hello Gnuradio experts,

I'm following the gnuradio tutorial for creating an out of tree python block.
My intention is fairly simple.  To have a input stream of 4 vectors
and average these 4 vectors, outputting a single average vector
after the 4th vector arrives.  Right now my vectors are 1024 channel
total power spectra.

My code is attached, along with the quality control code.
I've been at this for a while, but I must have the wrong model
in my head on how gnuradio holds these samples.

I'm not getting the proper size matches.   I can not find any
decimation help, despite hours of googling...

Suggestions?

THanks

Glen
  






My intention is to make this 4 vector average with decimate a
part of the .grc environment, but the problems are easier to diagnose,
as the QA code is stands alone.   Here are the messages I’m getting:


ESMU:vdecimate glangsto$ python qa_vdecimate.py
handler caught exception: could not broadcast input array from shape (4,1024) 
into shape (1,1024)
Traceback (most recent call last):
   File 
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py",
 line 55, in eval
 try: self._callback()
   File 
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py",
 line 160, in __gr_block_handle
 ) for i in self.__out_indexes],
   File "/Users/glangsto/Desktop/Research/vdecimate/vdecimate.py", line 58, in 
work
 out[:] = self.sum/float(self.count)
ValueError: could not broadcast input array from shape (4,1024) into shape 
(1,1024)
thread[thread-per-block[1]: ]: SWIG director method error. 
Error detected when calling 'feval_ll.eval'


I realize that your ultimate goal is to do a window=4 median filter, and this vecimate.py 
is a "stepping stone" to that.

But the specfic goal of "average 4 vectors, then decimate" can be achieved 
(approximately) with a single-pole IIR filter with the right parameters, followed
   by a keep-one-in-N block.


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio



___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Trouble decimating after averaging 4 vectors? Help?

[Glen I Langston]

Thanks Jeff,

I’ll give that a try.

I think I’m also missing something that tells gnu radio what the expected 
decimation is (?),
beyond the decimate = 4 parameter(?).

Also concerning your point, then if I have a non-multiple of 4 vectors I guess 
I need
to keep the sum and count, but I guess that is already a part of the code.

Best regards

Glen




> On Mar 11, 2018, at 3:54 PM, Jeff Long  wrote:
> 
> in[0] is not the first vector, it is all of the available input for port 0. 
> So, you feed in 4 vectors in your test program, and the shape of in[0] is 
> (4,1024). You're missing a loop that iterates over the input vectors.
> 
> On 03/11/2018 02:06 PM, Glen I Langston wrote:
>> Hello Gnuradio experts,
>> I'm following the gnuradio tutorial for creating an out of tree python block.
>> My intention is fairly simple.  To have a input stream of 4 vectors
>> and average these 4 vectors, outputting a single average vector
>> after the 4th vector arrives.  Right now my vectors are 1024 channel
>> total power spectra.
>> My code is attached, along with the quality control code.
>> I've been at this for a while, but I must have the wrong model
>> in my head on how gnuradio holds these samples.
>> I'm not getting the proper size matches.   I can not find any
>> decimation help, despite hours of googling...
>> Suggestions?
>> THanks
>> Glen
>> My intention is to make this 4 vector average with decimate a
>> part of the .grc environment, but the problems are easier to diagnose,
>> as the QA code is stands alone.   Here are the messages I’m getting:
>> ESMU:vdecimate glangsto$ python qa_vdecimate.py
>> handler caught exception: could not broadcast input array from shape 
>> (4,1024) into shape (1,1024)
>> Traceback (most recent call last):
>>   File 
>> "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py",
>>  line 55, in eval
>> try: self._callback()
>>   File 
>> "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py",
>>  line 160, in __gr_block_handle
>> ) for i in self.__out_indexes],
>>   File "/Users/glangsto/Desktop/Research/vdecimate/vdecimate.py", line 58, 
>> in work
>> out[:] = self.sum/float(self.count)
>> ValueError: could not broadcast input array from shape (4,1024) into shape 
>> (1,1024)
>> thread[thread-per-block[1]: ]: SWIG director method 
>> error. Error detected when calling 'feval_ll.eval'
>> ___
>> Discuss-gnuradio mailing list
>> Discuss-gnuradio@gnu.org
>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Trouble decimating after averaging 4 vectors? Help?

[Glen I Langston]

Hi Marcus,

Yes, the suggested combination of IIR filter and the keen one in N Block
would potentially do the job.   However as you pointed out, my end
goal is to have a median of 4 vectors block.  Also I was thinking that
a direct average would be numerically less expensive than the combination
of two different blocks.

Also, I’ve been trying for
a few weeks to get an out of tree module to work, so am stubbornly
hoping that I might figure it out.  (Actually I’m hoping someone might
tell me how to figure it out.)   

Your suggestion of returning 1 instead of vector size might solve this.
I’ll give it a try.

Best regards

Glen

> On Mar 11, 2018, at 3:23 PM, Marcus D. Leech  wrote:
> 
> On 03/11/2018 02:06 PM, Glen I Langston wrote:
>> Hello Gnuradio experts,
>> 
>> I'm following the gnuradio tutorial for creating an out of tree python block.
>> My intention is fairly simple.  To have a input stream of 4 vectors
>> and average these 4 vectors, outputting a single average vector
>> after the 4th vector arrives.  Right now my vectors are 1024 channel
>> total power spectra.
>> 
>> My code is attached, along with the quality control code.
>> I've been at this for a while, but I must have the wrong model
>> in my head on how gnuradio holds these samples.
>> 
>> I'm not getting the proper size matches.   I can not find any
>> decimation help, despite hours of googling...
>> 
>> Suggestions?
>> 
>> THanks
>> 
>> Glen
>>  
>> 
>> 
>> 
>> 
>> 
>> My intention is to make this 4 vector average with decimate a
>> part of the .grc environment, but the problems are easier to diagnose,
>> as the QA code is stands alone.   Here are the messages I’m getting:
>> 
>> 
>> ESMU:vdecimate glangsto$ python qa_vdecimate.py 
>> handler caught exception: could not broadcast input array from shape 
>> (4,1024) into shape (1,1024)
>> Traceback (most recent call last):
>>   File 
>> "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py",
>>  line 55, in eval
>> try: self._callback()
>>   File 
>> "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py",
>>  line 160, in __gr_block_handle
>> ) for i in self.__out_indexes],
>>   File "/Users/glangsto/Desktop/Research/vdecimate/vdecimate.py", line 58, 
>> in work
>> out[:] = self.sum/float(self.count)
>> ValueError: could not broadcast input array from shape (4,1024) into shape 
>> (1,1024)
>> thread[thread-per-block[1]: ]: SWIG director method 
>> error. Error detected when calling 'feval_ll.eval'
>> 
> I realize that your ultimate goal is to do a window=4 median filter, and this 
> vecimate.py is a "stepping stone" to that.
> 
> But the specfic goal of "average 4 vectors, then decimate" can be achieved 
> (approximately) with a single-pole IIR filter with the right parameters, 
> followed
>   by a keep-one-in-N block.
> 
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Trouble decimating after averaging 4 vectors? Help?

[Jeff Long]

in[0] is not the first vector, it is all of the available input for port 
0. So, you feed in 4 vectors in your test program, and the shape of 
in[0] is (4,1024). You're missing a loop that iterates over the input 
vectors.


On 03/11/2018 02:06 PM, Glen I Langston wrote:

Hello Gnuradio experts,

I'm following the gnuradio tutorial for creating an out of tree python 
block.

My intention is fairly simple.  To have a input stream of 4 vectors
and average these 4 vectors, outputting a single average vector
after the 4th vector arrives.  Right now my vectors are 1024 channel
total power spectra.

My code is attached, along with the quality control code.
I've been at this for a while, but I must have the wrong model
in my head on how gnuradio holds these samples.

I'm not getting the proper size matches.   I can not find any
decimation help, despite hours of googling...

Suggestions?

THanks

Glen






My intention is to make this 4 vector average with decimate a
part of the .grc environment, but the problems are easier to diagnose,
as the QA code is stands alone.   Here are the messages I’m getting:


ESMU:vdecimate glangsto$ python qa_vdecimate.py
handler caught exception: could not broadcast input array from shape 
(4,1024) into shape (1,1024)

Traceback (most recent call last):
   File 
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py", 
line 55, in eval

     try: self._callback()
   File 
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py", 
line 160, in __gr_block_handle

     ) for i in self.__out_indexes],
   File "/Users/glangsto/Desktop/Research/vdecimate/vdecimate.py", line 
58, in work

     out[:] = self.sum/float(self.count)
ValueError: could not broadcast input array from shape (4,1024) into 
shape (1,1024)
thread[thread-per-block[1]: ]: SWIG director method 
error. Error detected when calling 'feval_ll.eval'





___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio



___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Trouble decimating after averaging 4 vectors? Help?

[Marcus D. Leech]

On 03/11/2018 02:06 PM, Glen I Langston wrote:

Hello Gnuradio experts,

I'm following the gnuradio tutorial for creating an out of tree python 
block.

My intention is fairly simple.  To have a input stream of 4 vectors
and average these 4 vectors, outputting a single average vector
after the 4th vector arrives.  Right now my vectors are 1024 channel
total power spectra.

My code is attached, along with the quality control code.
I've been at this for a while, but I must have the wrong model
in my head on how gnuradio holds these samples.

I'm not getting the proper size matches.   I can not find any
decimation help, despite hours of googling...

Suggestions?

THanks

Glen






My intention is to make this 4 vector average with decimate a
part of the .grc environment, but the problems are easier to diagnose,
as the QA code is stands alone.   Here are the messages I’m getting:


ESMU:vdecimate glangsto$ python qa_vdecimate.py
handler caught exception: could not broadcast input array from shape 
(4,1024) into shape (1,1024)

Traceback (most recent call last):
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py", 
line 55, in eval

try: self._callback()
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py", 
line 160, in __gr_block_handle

) for i in self.__out_indexes],
  File "/Users/glangsto/Desktop/Research/vdecimate/vdecimate.py", line 
58, in work

out[:] = self.sum/float(self.count)
ValueError: could not broadcast input array from shape (4,1024) into 
shape (1,1024)
thread[thread-per-block[1]: ]: SWIG director 
method error. Error detected when calling 'feval_ll.eval'



What happens if you return 1 in the case where you're returning 
data? Don't forget, that's the "number of items", and your "items" 
are 1024-entry vectors.




___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Trouble decimating after averaging 4 vectors? Help?

[Marcus D. Leech]

On 03/11/2018 02:06 PM, Glen I Langston wrote:

Hello Gnuradio experts,

I'm following the gnuradio tutorial for creating an out of tree python 
block.

My intention is fairly simple.  To have a input stream of 4 vectors
and average these 4 vectors, outputting a single average vector
after the 4th vector arrives.  Right now my vectors are 1024 channel
total power spectra.

My code is attached, along with the quality control code.
I've been at this for a while, but I must have the wrong model
in my head on how gnuradio holds these samples.

I'm not getting the proper size matches.   I can not find any
decimation help, despite hours of googling...

Suggestions?

THanks

Glen






My intention is to make this 4 vector average with decimate a
part of the .grc environment, but the problems are easier to diagnose,
as the QA code is stands alone.   Here are the messages I’m getting:


ESMU:vdecimate glangsto$ python qa_vdecimate.py
handler caught exception: could not broadcast input array from shape 
(4,1024) into shape (1,1024)

Traceback (most recent call last):
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py", 
line 55, in eval

try: self._callback()
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/gr/gateway.py", 
line 160, in __gr_block_handle

) for i in self.__out_indexes],
  File "/Users/glangsto/Desktop/Research/vdecimate/vdecimate.py", line 
58, in work

out[:] = self.sum/float(self.count)
ValueError: could not broadcast input array from shape (4,1024) into 
shape (1,1024)
thread[thread-per-block[1]: ]: SWIG director 
method error. Error detected when calling 'feval_ll.eval'


I realize that your ultimate goal is to do a window=4 median filter, and 
this vecimate.py is a "stepping stone" to that.


But the specfic goal of "average 4 vectors, then decimate" can be 
achieved (approximately) with a single-pole IIR filter with the right 
parameters, followed

  by a keep-one-in-N block.


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

[Discuss-gnuradio] Trouble decimating after averaging 4 vectors? Help?

[Glen I Langston]

Hello Gnuradio experts,I'm following the gnuradio tutorial for creating an out of tree python block.My intention is fairly simple.  To have a input stream of 4 vectorsand average these 4 vectors, outputting a single average vectorafter the 4th vector arrives.  Right now my vectors are 1024 channeltotal power spectra.My code is attached, along with the quality control code.I've been at this for a while, but I must have the wrong modelin my head on how gnuradio holds these samples.I'm not getting the proper size matches.   I can not find anydecimation help, despite hours of googling...Suggestions?THanksGlen 
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# 
# Copyright 2018 <+YOU OR YOUR COMPANY+>.
# 
# This is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 3, or (at your option)
# any later version.
# 
# This software is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with this software; see the file COPYING.  If not, write to
# the Free Software Foundation, Inc., 51 Franklin Street,
# Boston, MA 02110-1301, USA.
# 

from gnuradio import gr, gr_unittest
from gnuradio import blocks
from vdecimate import vdecimate
import numpy as np

class qa_vdecimate (gr_unittest.TestCase):

def setUp (self):
self.tb = gr.top_block ()

def tearDown (self):
self.tb = None

def test_001_t (self):
# set up fg
vectorsize = 1024
nave = 4
xin = np.zeros(vectorsize)
xbig = np.zeros(4*vectorsize)

for iii in range(vectorsize):
xin[iii] = float(iii)
xbig[iii] = float(iii)
xbig[iii+vectorsize] = float(iii)
xbig[iii+(2*vectorsize)] = float(iii)
xbig[iii+(3*vectorsize)] = float(iii)
expected_result = xin

src = blocks.vector_source_f(xbig, False, vectorsize)
vdec = vdecimate( vectorsize, nave)
snk = blocks.vector_sink_f( vectorsize)
self.tb.connect( src, vdec)
self.tb.connect( vdec, snk)
self.tb.run ()
# check data
result_data = snk.data()
self.assertFloatTuplesAlmostEqual (expected_result, result_data, 6)

if __name__ == '__main__':
gr_unittest.run(qa_vdecimate, "qa_vdecimate.xml")
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# 
# Copyright 2018 <+YOU OR YOUR COMPANY+>.
# 
# This is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 3, or (at your option)
# any later version.
# 
# This software is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with this software; see the file COPYING.  If not, write to
# the Free Software Foundation, Inc., 51 Franklin Street,
# Boston, MA 02110-1301, USA.
# 

import numpy
from gnuradio import gr

class vdecimate(gr.decim_block):
"""
Vector decimate any size of input docstring for block vdecimate
"""
def __init__(self, vectorsize=1024, nave=4):
gr.decim_block.__init__(self,
"vdecimate",
[(numpy.float32, vectorsize)],
[(numpy.float32, vectorsize)],
nave)
self.vectorsize = int(vectorsize)
self.sum = numpy.zeros(self.vectorsize)
self.count = 0
self.nave = 0

def work(self, input_items, output_items):
in0 = input_items[0]
nin = len(in0)
# if change in the vector size, must restart sum
if (nin != self.vectorsize) or (self.count == 0):
self.vectorsize = nin
self.sum = numpy.zeros(nin)
# initialize sum and count
self.sum = in0
self.count = 1
else:
self.sum = self.sum + in0
self.count = self.count + 1

#if time to output vector
if self.count >= self.nave:

out = output_items[0]
out[:] = self.sum/float(self.count)
self.count = 0
return len(out[:])
else:
return 0


My intention is to make this 4 vector average with decimate apart of the .grc environment, but the problems are easier to diagnose,as the QA code is stands alone.   Here are the messages I’m getting:ESMU:vdecimate glangsto$ python qa_vdecimate.py handler caught exception: could