Re: GNURadio Leaking memory during repeated simulations
It's possible I suppose, but the IDE wouldn't have been my first guess, unless it's running in some kind of debugging mode where it can't garbage collect freed memory? I haven't used PyCharm so can't comment specifically on how it works. I would try to just to run the python example script from a run_gr.bat command line without anything else involved at all and see what happens. I ran the sample using VS 2015 Community (which I start ["devenv.exe"] from the run_gr.bat command line so that not only are the python environment vars included, but all the $PATH changes as well). Geof On Tue, Feb 18, 2020 at 12:40 PM Roman A Sandler wrote: > Him Geof, > > I am running GR using the standard install but from PyCharm (I imported > all the necessary environment variables from run_gr.bat into PyCharm > myself). I installed tqdm on top of the GR python via pip. I also confirmed > that the issue occurs even without tqdm. Could PyCharm be causing these > problems? > > -Roman > > On Thu, Feb 13, 2020 at 7:10 PM Geof Nieboer > wrote: > >> Roman, >> >> I was able to run your code, and got a consistent 150-160 it/s through >> the whole run, with about 65MB of memory in use (as reporting by Task >> Manager). This was on Windows 10 running GR 3.8.0.0. >> >> I noticed there was another package installed (tqdm) that's not part of >> the GR install. So I want to confirm... did you add that package to the >> python that installed as part of GR, or are you using a different python? >> >> If you are using a different python install, a possible explanation is >> compilers; the GR packages and binaries are all built with VS 2015, but the >> py2.7 standard is VS 2008 (which won't build GR any more), and mixing (VS) >> compilers can cause odd issues. So for GR, the entire python 2.7 and every >> package is built from scratch w/ 2015. tqdm doesn't seem to be affected, I >> added that to the GR python install using a pip install, nothing fancy, and >> it seems to have worked. >> >> If you are, however, running the script with the GR-installed python >> after opening with run_gr.bat, then I'm scratching my head what's happening. >> >> Geof >> >> On Thu, Feb 13, 2020 at 12:00 PM Roman A Sandler >> wrote: >> >>> Hi Marcus, >>> >>> Thanks for the reply! >>> >>> My GNURadio version: *3.8.0.0 (Python 2.7.10)* >>> It is the Windows 3.8.0.0 version downloaded from: >>> http://www.gcndevelopment.com/gnuradio/downloads.htm >>> >>> Complete reproducible code is below. I use the tqdm package to monitor >>> iterations per second. On my PC, the it/sec declines very precipitously >>> (starts at 85it/sec, then down to 22it/s after 40s and keeps dropping. >>> Eventually as low as 1 it/sec). >>> >>> >>> ``` >>> >>> import numpy as np >>> from gnuradio import gr, gr_unittest >>> from gnuradio import blocks >>> from gnuradio import digital >>> from gnuradio.filter import firdes >>> from gnuradio import channels >>> from tqdm import tqdm >>> >>> >>> def decode(channel_sigs): >>> tb = gr.top_block() >>> >>> ## >>> # Variables >>> ## >>> nfilts = 32 >>> sps = 4 >>> timing_loop_bw = 3 * 6.28 / 100.0 # NOTE: this controls convergence >>> speed! >>> constellation_scheme = digital.constellation_8psk().base() >>> rrc_taps = firdes.root_raised_cosine(nfilts, nfilts, 1.0 / float(sps), >>> 0.35, 11 * sps * nfilts) >>> >>> Actual blocks >>> channel_src = blocks.vector_source_c(channel_sigs, False) >>> digital_pfb_clock_sync = digital.pfb_clock_sync_ccf(sps, >>> timing_loop_bw, rrc_taps, nfilts, >>> nfilts / 2, 1.5, 1) >>> constellation_soft_decoder = >>> digital.constellation_soft_decoder_cf(constellation_scheme) >>> binary_slicer = digital.binary_slicer_fb() >>> blocks_char_to_float = blocks.char_to_float(1, 1) >>> recovered_bits_dst = blocks.vector_sink_f() >>> >>> ## >>> # Connections >>> ## >>> tb.connect((channel_src, 0), (digital_pfb_clock_sync, 0)) >>> tb.connect((digital_pfb_clock_sync, 0), (constellation_soft_decoder, 0)) >>> tb.connect((constellation_soft_decoder, 0), (binary_slicer, 0)) >>> tb.connect((binary_slicer, 0), (blocks_char_to_float, 0)) >>> tb.connect((blocks_char_to_float, 0), (recovered_bits_dst, 0)) >>> >>> tb.run() >>> recovered_bits = np.array(recovered_bits_dst.data()) >>> >>> return recovered_bits >>> >>> >>> if __name__ == '__main__': >>> n_trls = 1 >>> n_samples = >>> sig = np.random.normal(size=(n_samples,)) + 1j * >>> np.random.normal(size=(n_samples,)) >>> >>> for n in tqdm(range(n_trls)): >>> decode(sig) >>> >>> ``` >>> >>> >>> On Thu, Feb 13, 2020 at 2:11 AM Müller, Marcus (CEL) >>> wrote: >>> Hi, huh. That
Re: Using ZMQ source blocks with a separate process
I have a freshly built GR 3.8.1.0-rc1 and GRC 3.8.1.0 on rPi. A simple flowgraph of ZMQ PULL Message Source feeding a Message Debug gives the following errors: ``` Executing: /usr/bin/python3 -u /home/pi/GRdev/msg_test.py libEGL warning: DRI2: failed to authenticate qt5ct: using qt5ct plugin /home/pi/.gnuradio/prefs/vmcircbuf_default_factory: No such file or directory vmcircbuf_createfilemapping: createfilemapping is not available gr::vmcircbuf_sysv_shm: shmat (2): Invalid argument gr::vmcircbuf_sysv_shm: shmat (2): Invalid argument gr::vmcircbuf_sysv_shm: shmat (2): Invalid argument ``` /home/pi/.gnuradio/prefs/vmcircbuf_default_factory contains: gr::vmcircbuf_mmap_shm_open_factory --- Barry Duggan KV4FV On 2020-02-18 09:18, Michael Dickens wrote: GR's ZMQ should work across threads, processes, and networks; between GR and non-GR so long as the packet formatting is correct. I've used ZMQ between GR within a network, never done this between GR and non-GR though ... so maybe this is an aspiration rather than guarantee? - MLD On Mon, Feb 17, 2020 at 9:07 PM Barry Duggan wrote: Hi, I would like to use one of the ZMQ source blocks (such as ZMQ PULL Message Source) to communicate with a separate Python process. I have been able to use various Python examples to test between processes, but when I try to use a GRC block, it fails. Are there restrictions which prohibit data exchange outside the gnuradio realm? Thanks for your help. -- Barry Duggan KV4FV -- Michael Dickens Ettus Research Technical Support Email: supp...@ettus.com Web: https://ettus.com/
Re: GNURadio Leaking memory during repeated simulations
Him Geof, I am running GR using the standard install but from PyCharm (I imported all the necessary environment variables from run_gr.bat into PyCharm myself). I installed tqdm on top of the GR python via pip. I also confirmed that the issue occurs even without tqdm. Could PyCharm be causing these problems? -Roman On Thu, Feb 13, 2020 at 7:10 PM Geof Nieboer wrote: > Roman, > > I was able to run your code, and got a consistent 150-160 it/s through the > whole run, with about 65MB of memory in use (as reporting by Task > Manager). This was on Windows 10 running GR 3.8.0.0. > > I noticed there was another package installed (tqdm) that's not part of > the GR install. So I want to confirm... did you add that package to the > python that installed as part of GR, or are you using a different python? > > If you are using a different python install, a possible explanation is > compilers; the GR packages and binaries are all built with VS 2015, but the > py2.7 standard is VS 2008 (which won't build GR any more), and mixing (VS) > compilers can cause odd issues. So for GR, the entire python 2.7 and every > package is built from scratch w/ 2015. tqdm doesn't seem to be affected, I > added that to the GR python install using a pip install, nothing fancy, and > it seems to have worked. > > If you are, however, running the script with the GR-installed python after > opening with run_gr.bat, then I'm scratching my head what's happening. > > Geof > > On Thu, Feb 13, 2020 at 12:00 PM Roman A Sandler > wrote: > >> Hi Marcus, >> >> Thanks for the reply! >> >> My GNURadio version: *3.8.0.0 (Python 2.7.10)* >> It is the Windows 3.8.0.0 version downloaded from: >> http://www.gcndevelopment.com/gnuradio/downloads.htm >> >> Complete reproducible code is below. I use the tqdm package to monitor >> iterations per second. On my PC, the it/sec declines very precipitously >> (starts at 85it/sec, then down to 22it/s after 40s and keeps dropping. >> Eventually as low as 1 it/sec). >> >> >> ``` >> >> import numpy as np >> from gnuradio import gr, gr_unittest >> from gnuradio import blocks >> from gnuradio import digital >> from gnuradio.filter import firdes >> from gnuradio import channels >> from tqdm import tqdm >> >> >> def decode(channel_sigs): >> tb = gr.top_block() >> >> ## >> # Variables >> ## >> nfilts = 32 >> sps = 4 >> timing_loop_bw = 3 * 6.28 / 100.0 # NOTE: this controls convergence >> speed! >> constellation_scheme = digital.constellation_8psk().base() >> rrc_taps = firdes.root_raised_cosine(nfilts, nfilts, 1.0 / float(sps), >> 0.35, 11 * sps * nfilts) >> >> Actual blocks >> channel_src = blocks.vector_source_c(channel_sigs, False) >> digital_pfb_clock_sync = digital.pfb_clock_sync_ccf(sps, timing_loop_bw, >> rrc_taps, nfilts, >> nfilts / 2, 1.5, 1) >> constellation_soft_decoder = >> digital.constellation_soft_decoder_cf(constellation_scheme) >> binary_slicer = digital.binary_slicer_fb() >> blocks_char_to_float = blocks.char_to_float(1, 1) >> recovered_bits_dst = blocks.vector_sink_f() >> >> ## >> # Connections >> ## >> tb.connect((channel_src, 0), (digital_pfb_clock_sync, 0)) >> tb.connect((digital_pfb_clock_sync, 0), (constellation_soft_decoder, 0)) >> tb.connect((constellation_soft_decoder, 0), (binary_slicer, 0)) >> tb.connect((binary_slicer, 0), (blocks_char_to_float, 0)) >> tb.connect((blocks_char_to_float, 0), (recovered_bits_dst, 0)) >> >> tb.run() >> recovered_bits = np.array(recovered_bits_dst.data()) >> >> return recovered_bits >> >> >> if __name__ == '__main__': >> n_trls = 1 >> n_samples = >> sig = np.random.normal(size=(n_samples,)) + 1j * >> np.random.normal(size=(n_samples,)) >> >> for n in tqdm(range(n_trls)): >> decode(sig) >> >> ``` >> >> >> On Thu, Feb 13, 2020 at 2:11 AM Müller, Marcus (CEL) >> wrote: >> >>> Hi, >>> >>> huh. That looks hard to debug; also, the slow down is suspicious (as >>> long as there's memory available, it shouldn't take significantly >>> longer to get some – usually, memory fragmentation isn't *that* bad, >>> and this shouldn't be doing *that* much memory allocation). >>> >>> Could you put all your code in one .py file (or a set of these) that >>> one can simply execute right away? That would allow us to reproduce. >>> Also, could you tell us your specific GNU Radio version (all four >>> digits of it?). >>> >>> Best regards, >>> Marcus >>> >>> On Tue, 2020-02-11 at 16:34 -0800, Roman A Sandler wrote: >>> > Hi, >>> > >>> > I am using GNURadio to decode a large amount of 1024-sample complex >>> > vectors of different modulation schemes. Thus, I have a for loop >>> > which runs
Re: Using ZMQ source blocks with a separate process
Hi Barry, >> I have >> been able to use various Python examples to test between processes, but >> when I try to use a GRC block, it fails. Are there restrictions which >> prohibit data exchange outside the gnuradio realm? >> Make sure to use the same ZMQ version as is used by GNURadio. Cheers Christoph
Re: Using ZMQ source blocks with a separate process
GR's ZMQ should work across threads, processes, and networks; between GR and non-GR so long as the packet formatting is correct. I've used ZMQ between GR within a network, never done this between GR and non-GR though ... so maybe this is an aspiration rather than guarantee? - MLD On Mon, Feb 17, 2020 at 9:07 PM Barry Duggan wrote: > Hi, > > I would like to use one of the ZMQ source blocks (such as ZMQ PULL > Message Source) to communicate with a separate Python process. I have > been able to use various Python examples to test between processes, but > when I try to use a GRC block, it fails. Are there restrictions which > prohibit data exchange outside the gnuradio realm? > > Thanks for your help. > -- > Barry Duggan KV4FV > > -- Michael Dickens Ettus Research Technical Support Email: supp...@ettus.com Web: https://ettus.com/
