Re: [Discuss-gnuradio] SDRplay rewrite w/RSP2
The RSP1A and RSP2 work well for me using this code, and there are many improvements since my last post, so I'm looking for other people to help test it. Something along the lines of the following commands will allow installation and test using a temporary directory: git clone https://github.com/willcode/gr-osmosdr cd gr-osmosdr git checkout sdrplay2 mkdir build cd build cmake -DCMAKE_INSTALL_PREFIX=/tmp/test -DENABLE_NONFREE=yes .. make mkdir /tmp/test make install LD_LIBRARY_PATH=/tmp/test/lib64:$LD_LIBRARY_PATH gqrx This product does require a non-free library that gets compiled in to gr-osmosdr. This limits inclusion on distributions, but it's still easy to use from a PyBOMBs installation or a manual install, as above. On 12/16/2017 04:54 PM, Jeff Long wrote: I picked up a SDRplay RSP2 recently, and the gr-osmosdr support needed some work. Here's a rewrite that works with RSP2 and has other improvements. When the RSP1A comes in, I'll finish support for that. No plans to pick up a RSP1, but someone can see if this works if interested. This was tested against the Nov 2017 binary blob. https://github.com/willcode/gr-osmosdr/tree/sdrplay2 = Complete rewrite of sdrplay support - Supports RSP2 - Will support RSP1A when hardware arrives - RSP1 is untested - Uses streaming callbacks instead of polling - Tuning, gain control imporoved, AGC supported - Better performance (one less layer of buffers) ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] pmt.to_python behaves differently in console and in gnuradio
Nevermind.Problem was in my code creating the np.array over on the send side. Sorry to waste the bandwidth. -- Tom, N5EG On Mon, Jan 1, 2018 at 1:23 PM, Tom McDermottwrote: > > Hello, Happy New Year ! > > I am trying to use message passing in gnuradio maint branch, 3.7.11.1 > (python 2.7 based) > > When just bringing up a command line window, starting python 2.7, > importing gnuradio, numpy and pmt, > the behavior I get (which agrees with the documentation): > > n = np.array([1,2,3,4,5], dtype=uint8) > p = pmt.to_pmt(n) > r = pmt.to_python(p) > > printing the types of a, p, and r yield the expected... > > type(n) --> > type(p) --> > type(r) --> > > > > However when in gnuradio > create and pass the pmt via the message port, > I get a string type after converting to python: > > > def handle_rxmsg(self, msg): > dat = pmt.to_python(msg) # dat is an np.array[9] of type np.uint8 > print "Type of msg: ", type(msg) > print "Type of dat: ", type(dat) > > > This prints on the gnuradio console: > > Type of msg: > Type of dat: > > > Does anyone know how do I get back a numpy array from pmt while > running under gnuradio? > > Why does this work in a command line console, but not in gnuradio? > > > -- Tom, N5EG > > > > ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
[Discuss-gnuradio] pmt.to_python behaves differently in console and in gnuradio
Hello, Happy New Year ! I am trying to use message passing in gnuradio maint branch, 3.7.11.1 (python 2.7 based) When just bringing up a command line window, starting python 2.7, importing gnuradio, numpy and pmt, the behavior I get (which agrees with the documentation): n = np.array([1,2,3,4,5], dtype=uint8) p = pmt.to_pmt(n) r = pmt.to_python(p) printing the types of a, p, and r yield the expected... type(n) --> type(p) --> type(r) --> However when in gnuradio create and pass the pmt via the message port, I get a string type after converting to python: def handle_rxmsg(self, msg): dat = pmt.to_python(msg) # dat is an np.array[9] of type np.uint8 print "Type of msg: ", type(msg) print "Type of dat: ", type(dat) This prints on the gnuradio console: Type of msg: Type of dat: Does anyone know how do I get back a numpy array from pmt while running under gnuradio? Why does this work in a command line console, but not in gnuradio? -- Tom, N5EG ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] Serious bug in tag propagation with non-integer relative rate
(and if I'd actually read Andy's last post carefully ...) On 01/01/2018 01:32 PM, Jeff Long wrote: As Eugene suggests, the fractional resampler can be handled by doing a linear mapping from the input to the output of a single work call. Wouldn't this work with most or all blocks? There would also need to be an offset in order to handle history. Then, there wouldn't be a need to use higher precision counters. On 01/01/2018 11:24 AM, Jeff Long wrote: I think the reason rate changes are difficult for tags is that the block executor tries to guess what a block is doing based on its long-term or static rate change. The block itself knows exactly how input items relate to output items, and maybe certain blocks need to provide more feedback than just a rate. In those cases, the executor could use the exact translation provided by the block, and in tamer cases the rate can still be used. On 01/01/2018 10:53 AM, Andy Walls wrote: Hi Marcus: On Sun, 2017-12-31 at 15:09 +, Müller, Marcus (CEL) wrote: Hi Andy, hi Eugene Hm, coming back to an idea I had not so long ago: tag offset should not be 64bit unsihned integers only, but also have a 64 bit fractional part. That would not immediately solve the computational inaccurracy during resampling, but would at least for multi (as in >2) rate systems (eg. rational resampling for sample rate reduction, arbitrary resampling for clock recovery) at least give the option of having consistent tagging. IMO, adding a fractional part does not solve the problem at hand - correct tag propagation through 1 block - but it probably is required for precise propagation through multiple rate changing blocks. I'm keeping it in mind, but I want to avoid changing existing interfaces right now. I.e. get_tags_in_[range|window]() runs into a backward compatability issue for a tag that comes < 0.5 samples before the beginning of the requested range: should the tag offset -0.49 samples realtive to the start of the window be returned or not? Personally, my gut feeling is that for this kind of divisional math, floating point is not the number format of choice Agreed. (for multiple (as in 2) reasons). This actually doesn't apply to the arbitrary resampling case as much, but the d_relative_rate property of a block shouldn't be a floating point, Internal to the gnuradio-runtime there are a number of places where, for estimating, using a floating point d_relative_rate is just fine. It is expedient and avoids the problem of interger overflow for cases where one has large integer interpolation and decimation factors. but a ratio of two integers, one of them being signed. Maybe I should be adding a set_relative_rate(int64_t numerator, uint_64_t denominator) to block, and make set_relative_rate(double) a wrapper for that (and of course change the block_executor to do fractional rather than floating point math); So that's what my change does. :) I also changed all the blocks, that could easily easily call the integer version of set_relataive_rate(interpolation, decimation), to do so. To avoid integer overflow, I picked an algoritm in the new set_relative_rate(double) wrapper, to convert the relative rate to a ratio of two integers that got within 1 ppm of the passed in relative rate. Because sane users only use relative rates of 1/25 or 1001/1000 or 255/256 or 2/3 or something like that, and whacky relative rates using large (>16 bit) integers in the ratio are just a product of rounding, right? Well I was wrong. :( Unfortunately, the fractional resampler is a corner case that actually uses the rounded (in binary) version of the resampling rate, that the user specifies, in its actual operation. Computing exact integer ratios to represent the double can lead to 53 bit numbers (IIRC), which can lead to integer overflows when multiplying uint64_t's together. :( So what I'm left with are two broad options to deal with handling a double relative rate (like the fractional resampler uses): 1. Implement an method like Eugene suggests: do a double multiply, get a rounded result, do a double division to get back to an initial offset, subtract off that initial offset, do a double multiply again, and then a final add. or 2. Allow and compute large, exact integer ratios of interpolation/decimation rate and use multiple precision (128 bit) arithmetic when overflow out of 64 bits is possible. Maybe using boost::multiprescision with the MIPR library on the backend. http://www.boost.org/doc/libs/1_66_0/libs/multiprecision/doc/html/index .html http://mpir.org/ (MPIR is supposedly better for Windows compatability than GMP.) I'm leaning towards option 2. I find that I never calculate a rate other than by calculating the floating point approximation of a fraction of integers, and that there's always been subtle problems when running odd ratios for prolonged times. Technically a finite precision double can always be represented by a ratio of integers, but those
Re: [Discuss-gnuradio] Serious bug in tag propagation with non-integer relative rate
As Eugene suggests, the fractional resampler can be handled by doing a linear mapping from the input to the output of a single work call. Wouldn't this work with most or all blocks? There would also need to be an offset in order to handle history. Then, there wouldn't be a need to use higher precision counters. On 01/01/2018 11:24 AM, Jeff Long wrote: I think the reason rate changes are difficult for tags is that the block executor tries to guess what a block is doing based on its long-term or static rate change. The block itself knows exactly how input items relate to output items, and maybe certain blocks need to provide more feedback than just a rate. In those cases, the executor could use the exact translation provided by the block, and in tamer cases the rate can still be used. On 01/01/2018 10:53 AM, Andy Walls wrote: Hi Marcus: On Sun, 2017-12-31 at 15:09 +, Müller, Marcus (CEL) wrote: Hi Andy, hi Eugene Hm, coming back to an idea I had not so long ago: tag offset should not be 64bit unsihned integers only, but also have a 64 bit fractional part. That would not immediately solve the computational inaccurracy during resampling, but would at least for multi (as in >2) rate systems (eg. rational resampling for sample rate reduction, arbitrary resampling for clock recovery) at least give the option of having consistent tagging. IMO, adding a fractional part does not solve the problem at hand - correct tag propagation through 1 block - but it probably is required for precise propagation through multiple rate changing blocks. I'm keeping it in mind, but I want to avoid changing existing interfaces right now. I.e. get_tags_in_[range|window]() runs into a backward compatability issue for a tag that comes < 0.5 samples before the beginning of the requested range: should the tag offset -0.49 samples realtive to the start of the window be returned or not? Personally, my gut feeling is that for this kind of divisional math, floating point is not the number format of choice Agreed. (for multiple (as in 2) reasons). This actually doesn't apply to the arbitrary resampling case as much, but the d_relative_rate property of a block shouldn't be a floating point, Internal to the gnuradio-runtime there are a number of places where, for estimating, using a floating point d_relative_rate is just fine. It is expedient and avoids the problem of interger overflow for cases where one has large integer interpolation and decimation factors. but a ratio of two integers, one of them being signed. Maybe I should be adding a set_relative_rate(int64_t numerator, uint_64_t denominator) to block, and make set_relative_rate(double) a wrapper for that (and of course change the block_executor to do fractional rather than floating point math); So that's what my change does. :) I also changed all the blocks, that could easily easily call the integer version of set_relataive_rate(interpolation, decimation), to do so. To avoid integer overflow, I picked an algoritm in the new set_relative_rate(double) wrapper, to convert the relative rate to a ratio of two integers that got within 1 ppm of the passed in relative rate. Because sane users only use relative rates of 1/25 or 1001/1000 or 255/256 or 2/3 or something like that, and whacky relative rates using large (>16 bit) integers in the ratio are just a product of rounding, right? Well I was wrong. :( Unfortunately, the fractional resampler is a corner case that actually uses the rounded (in binary) version of the resampling rate, that the user specifies, in its actual operation. Computing exact integer ratios to represent the double can lead to 53 bit numbers (IIRC), which can lead to integer overflows when multiplying uint64_t's together. :( So what I'm left with are two broad options to deal with handling a double relative rate (like the fractional resampler uses): 1. Implement an method like Eugene suggests: do a double multiply, get a rounded result, do a double division to get back to an initial offset, subtract off that initial offset, do a double multiply again, and then a final add. or 2. Allow and compute large, exact integer ratios of interpolation/decimation rate and use multiple precision (128 bit) arithmetic when overflow out of 64 bits is possible. Maybe using boost::multiprescision with the MIPR library on the backend. http://www.boost.org/doc/libs/1_66_0/libs/multiprecision/doc/html/index .html http://mpir.org/ (MPIR is supposedly better for Windows compatability than GMP.) I'm leaning towards option 2. I find that I never calculate a rate other than by calculating the floating point approximation of a fraction of integers, and that there's always been subtle problems when running odd ratios for prolonged times. Technically a finite precision double can always be represented by a ratio of integers, but those integers can be very (2^53-ish) large. Thanks for the feedback. Regards, Andy Best regards, Marcus On
[Discuss-gnuradio] Boost 1.66.0 Compatibility: GNU Radio & Volk & UHD
In initial testing, Boost 1.66.0 is fully compatible with the current GNU Radio and Volk GIT master & current releases. There's an issue with UHD that has already been fixed in the GIT master, but the new boost otherwise seems to work with UHD & the backport to the latest release is very straight forward. Cheers! - MLD ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] Serious bug in tag propagation with non-integer relative rate
I think the reason rate changes are difficult for tags is that the block executor tries to guess what a block is doing based on its long-term or static rate change. The block itself knows exactly how input items relate to output items, and maybe certain blocks need to provide more feedback than just a rate. In those cases, the executor could use the exact translation provided by the block, and in tamer cases the rate can still be used. On 01/01/2018 10:53 AM, Andy Walls wrote: Hi Marcus: On Sun, 2017-12-31 at 15:09 +, Müller, Marcus (CEL) wrote: Hi Andy, hi Eugene Hm, coming back to an idea I had not so long ago: tag offset should not be 64bit unsihned integers only, but also have a 64 bit fractional part. That would not immediately solve the computational inaccurracy during resampling, but would at least for multi (as in >2) rate systems (eg. rational resampling for sample rate reduction, arbitrary resampling for clock recovery) at least give the option of having consistent tagging. IMO, adding a fractional part does not solve the problem at hand - correct tag propagation through 1 block - but it probably is required for precise propagation through multiple rate changing blocks. I'm keeping it in mind, but I want to avoid changing existing interfaces right now. I.e. get_tags_in_[range|window]() runs into a backward compatability issue for a tag that comes < 0.5 samples before the beginning of the requested range: should the tag offset -0.49 samples realtive to the start of the window be returned or not? Personally, my gut feeling is that for this kind of divisional math, floating point is not the number format of choice Agreed. (for multiple (as in 2) reasons). This actually doesn't apply to the arbitrary resampling case as much, but the d_relative_rate property of a block shouldn't be a floating point, Internal to the gnuradio-runtime there are a number of places where, for estimating, using a floating point d_relative_rate is just fine. It is expedient and avoids the problem of interger overflow for cases where one has large integer interpolation and decimation factors. but a ratio of two integers, one of them being signed. Maybe I should be adding a set_relative_rate(int64_t numerator, uint_64_t denominator) to block, and make set_relative_rate(double) a wrapper for that (and of course change the block_executor to do fractional rather than floating point math); So that's what my change does. :) I also changed all the blocks, that could easily easily call the integer version of set_relataive_rate(interpolation, decimation), to do so. To avoid integer overflow, I picked an algoritm in the new set_relative_rate(double) wrapper, to convert the relative rate to a ratio of two integers that got within 1 ppm of the passed in relative rate. Because sane users only use relative rates of 1/25 or 1001/1000 or 255/256 or 2/3 or something like that, and whacky relative rates using large (>16 bit) integers in the ratio are just a product of rounding, right? Well I was wrong. :( Unfortunately, the fractional resampler is a corner case that actually uses the rounded (in binary) version of the resampling rate, that the user specifies, in its actual operation. Computing exact integer ratios to represent the double can lead to 53 bit numbers (IIRC), which can lead to integer overflows when multiplying uint64_t's together. :( So what I'm left with are two broad options to deal with handling a double relative rate (like the fractional resampler uses): 1. Implement an method like Eugene suggests: do a double multiply, get a rounded result, do a double division to get back to an initial offset, subtract off that initial offset, do a double multiply again, and then a final add. or 2. Allow and compute large, exact integer ratios of interpolation/decimation rate and use multiple precision (128 bit) arithmetic when overflow out of 64 bits is possible. Maybe using boost::multiprescision with the MIPR library on the backend. http://www.boost.org/doc/libs/1_66_0/libs/multiprecision/doc/html/index .html http://mpir.org/ (MPIR is supposedly better for Windows compatability than GMP.) I'm leaning towards option 2. I find that I never calculate a rate other than by calculating the floating point approximation of a fraction of integers, and that there's always been subtle problems when running odd ratios for prolonged times. Technically a finite precision double can always be represented by a ratio of integers, but those integers can be very (2^53-ish) large. Thanks for the feedback. Regards, Andy Best regards, Marcus On Sat, 2017-12-30 at 18:24 -0500, Andy Walls wrote: Hi Eugene: On Wed, 2017-12-27 at 16:18 -0500, Andy Walls wrote: Hi Eugene From: Eugene Grayver Date: Thu, 9 Nov 2017 19:52:35 + There is a major problem with the way tags are propagated in blocks with non-integer relative rate. If the ratio is not a power of two, the numerical accuracy of the
Re: [Discuss-gnuradio] noutput_items in every block
Hi Sakthivel - Short answers: The value can vary for each call; it is determined by the scheduler. I've provided more info below if you're curious. Cheers! - MLD Details: One way to think of your questions is to imagine the finite-length I/O buffers that hold the data between blocks, and note that, in general, it is more CPU efficient to process "more" data than very small chunks -- typically 1k of data can be handled more efficiently than 4 bytes, when you consider the CPU overhead required for the scheduler (this is true up to some "large" data amount, when processing efficiency peaks and possible even drops somewhat off of peak). When the flowgraph starts, these buffers are all empty; so the scheduler tries to get blocks to process as much input data as possible. Once the flowgraph is running, the buffers hold (for all practical purposes) random amounts of data, which means that the blocks (in general) will not be able to process the amount of data as at startup time. Data will flow roughly in bursts from source to sink, but since each block is executing in its own thread the end result is data pipelining: "work" for any specific block happens when there is simultaneously "enough" input data and "enough" output buffer space -- combine the "note" above with this concept and you have a rough interpretation of the scheduler algorithm. Thus, with data streaming the scheduler has to be able to work with dynamic amounts of I/O data / buffer space. On Mon, Jan 1, 2018, at 10:30 AM, Sakthivel Velumani wrote: > I am new to GNU radio. I have this general doubt that when items are streamed > from one block to another, how many input_items per port (consider a type > general block) are passed when the work() function of the block is called > each time? I guess this is handled by the GNU radio's scheduler but would > like to know if this is constant or it varies for each call? ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] Serious bug in tag propagation with non-integer relative rate
Hi Marcus: On Sun, 2017-12-31 at 15:09 +, Müller, Marcus (CEL) wrote: > Hi Andy, hi Eugene > > Hm, coming back to an idea I had not so long ago: > > tag offset should not be 64bit unsihned integers only, but also have > a > 64 bit fractional part. > > That would not immediately solve the computational inaccurracy during > resampling, but would at least for multi (as in >2) rate systems (eg. > rational resampling for sample rate reduction, arbitrary resampling > for > clock recovery) at least give the option of having consistent > tagging. IMO, adding a fractional part does not solve the problem at hand - correct tag propagation through 1 block - but it probably is required for precise propagation through multiple rate changing blocks. I'm keeping it in mind, but I want to avoid changing existing interfaces right now. I.e. get_tags_in_[range|window]() runs into a backward compatability issue for a tag that comes < 0.5 samples before the beginning of the requested range: should the tag offset -0.49 samples realtive to the start of the window be returned or not? > Personally, my gut feeling is that for this kind of divisional math, > floating point is not the number format of choice Agreed. > (for multiple (as in > > 2) reasons). > > This actually doesn't apply to the arbitrary resampling case as much, > but the d_relative_rate property of a block shouldn't be a floating > point, Internal to the gnuradio-runtime there are a number of places where, for estimating, using a floating point d_relative_rate is just fine. It is expedient and avoids the problem of interger overflow for cases where one has large integer interpolation and decimation factors. > but a ratio of two integers, one of them being signed. Maybe I > should be adding a set_relative_rate(int64_t numerator, uint_64_t > denominator) to block, and make set_relative_rate(double) a wrapper > for > that (and of course change the block_executor to do fractional rather > than floating point math); So that's what my change does. :) I also changed all the blocks, that could easily easily call the integer version of set_relataive_rate(interpolation, decimation), to do so. To avoid integer overflow, I picked an algoritm in the new set_relative_rate(double) wrapper, to convert the relative rate to a ratio of two integers that got within 1 ppm of the passed in relative rate. Because sane users only use relative rates of 1/25 or 1001/1000 or 255/256 or 2/3 or something like that, and whacky relative rates using large (>16 bit) integers in the ratio are just a product of rounding, right? Well I was wrong. :( Unfortunately, the fractional resampler is a corner case that actually uses the rounded (in binary) version of the resampling rate, that the user specifies, in its actual operation. Computing exact integer ratios to represent the double can lead to 53 bit numbers (IIRC), which can lead to integer overflows when multiplying uint64_t's together. :( So what I'm left with are two broad options to deal with handling a double relative rate (like the fractional resampler uses): 1. Implement an method like Eugene suggests: do a double multiply, get a rounded result, do a double division to get back to an initial offset, subtract off that initial offset, do a double multiply again, and then a final add. or 2. Allow and compute large, exact integer ratios of interpolation/decimation rate and use multiple precision (128 bit) arithmetic when overflow out of 64 bits is possible. Maybe using boost::multiprescision with the MIPR library on the backend. http://www.boost.org/doc/libs/1_66_0/libs/multiprecision/doc/html/index .html http://mpir.org/ (MPIR is supposedly better for Windows compatability than GMP.) I'm leaning towards option 2. > I find that I never calculate a rate other > than by calculating the floating point approximation of a fraction of > integers, and that there's always been subtle problems when running > odd > ratios for prolonged times. Technically a finite precision double can always be represented by a ratio of integers, but those integers can be very (2^53-ish) large. Thanks for the feedback. Regards, Andy > Best regards, > Marcus > > On Sat, 2017-12-30 at 18:24 -0500, Andy Walls wrote: > > Hi Eugene: > > > > On Wed, 2017-12-27 at 16:18 -0500, Andy Walls wrote: > > > Hi Eugene > > > > > > > From: Eugene Grayver > > > > Date: Thu, 9 Nov 2017 19:52:35 + > > > > > > > > There is a major problem with the way tags are propagated in > > > > blocks > > > > with non-integer relative rate. If the ratio is not a power of > > > > two, > > > > the numerical accuracy of the floating point will cause the > > > > output > > > > tags to diverge from the input tags. Consider the fractional > > > > resampler. It accumulates the timing offset in the range of 0 > > > > to > > > > 1. > > > > However tag propagation multiplies the ratio by the sample > > > > number. > > > > As > > > > the sample number grows the LSB
[Discuss-gnuradio] noutput_items in every block
Hi all, I am new to GNU radio. I have this general doubt that when items are streamed from one block to another, how many input_items per port (consider a type general block) are passed when the work() function of the block is called each time? I guess this is handled by the GNU radio's scheduler but would like to know if this is constant or it varies for each call? Best Sakthivel ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio