Re: [Discuss-gnuradio] [GREP] Separate scheduler and GNU Radio base files
Miklos, thanks for the feedback. For your comments on the block executor, that's a great point -- where does it belong? At some point, we'll need owners for this task. Would you be interested in participating? -- M On Wed, Jan 2, 2019 at 5:54 PM Miklos Maroti wrote: > Dear Martin, > > I have been looking at the GNURadio scheduler quite a bit recently and > would like to summarize my experiences that could affect the direction > of this GREP. As you have properly observed, the scheduler is very > hard to change as it is, but there are new opportunities if we allow > changes. There are two components: scheduler itself (the scheduler_tpb > and tpb_thread_body) and the block executor. I think both needs some > attention, but it is not clear if you want to address only the > scheduler itself or together with the executor. > > - For the scheduler I would really like to introduce a type of > scheduler that manages only a subset of blocks for minimizing latency > by draining internal buffers within this group of block before new > data is accepted into these blocks. Think of a long sequence of blocks > processing packets that we would like to perform one after the other > without bringing all that functionality into a single block. Maybe the > flow graph need not be flattened completely, but certain hierarchical > blocks could be managed by their own scheduler (so flattened into a > level 1 hierarchy). Maybe the single threaded scheduler is just a > special case of this. > > - Certain tasks, such as combining packets into frames, might need to > know if future packets are arriving. This can be accomplished with a > timeout, so the block would be notified to flush its internal buffers > if the work function has not been called for a predefined time. There > is no such functionality within the current scheduler and I think it > would be possible to implement one by exposing the timeout used within > tpb_thread_body. > > - The block executor is also a very complex beast. I am wondering why > the functionality of the executor not implemented within the > basic_block where it could be overloaded. > > Hope this starts the discussion and others can voice their opinion. > > Best, > Miklos > > On Thu, Dec 27, 2018 at 10:58 PM Martin Braun wrote: > > > > Hi all, > > > > final GREP of the day: > https://github.com/gnuradio/greps/blob/master/grep-0016-separate-scheduler.md > > > > This is possible the most fundamental and influential GREPs that were > added so far. I would find it hard to find any reasons not to do this -- of > course, the question remains, who will do it. Any takers? I'm hoping that > by separating scheduler from base, we can open up the avenue for more and > better research into scheduling, as well as custom scheduler development. > > > > Discuss! > > > > -- Martin > > ___ > > Discuss-gnuradio mailing list > > Discuss-gnuradio@gnu.org > > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio > > ___ > Discuss-gnuradio mailing list > Discuss-gnuradio@gnu.org > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio > ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] [GREP] Remove log4cpp
Hi all, there's been a little discussion outside of this thread, and I'd like to consolidate all the points. Marcus, I and some others have also discussed some things on chat, too, and of course there's Jared's and Michael's comments here. Let me summarize: - We would like to drop any kind of dependency and hand-roll our logger. I'll make some points on why further down. - Any log message must have a severity and a "category" (or "source ID"). We already have this, but want to keep it. - The source ID for a block should be its ID. I like Jared's suggestion of the log-hierarchy following the block-hierarchy. - We'll also drop the debug logger vs the regular logger. Devs can use the DEBUG and TRACE log levels for their debugging. - We absolutely want to make sure that log messages are properly interleaved. This requires some amount of serialization/thread handling, I'll also comment on that later. - By default, there should be log files and logging to stderr. However, we also want to enable (or at least, not disable) other logging paths, such as journald, or network logging. This might be as simple as providing generic hooks for other loggers. - If possible, we would also like to pull in log messages from dependencies, so they all go through the same logging interface. For example, UHD log messages could be redirected to the GNU Radio logger when running gr-uhd applications, so UHD and GNU Radio don't clobber each other with different formats of their logging messages. Out of these requirements, there fall some design decisions: - There will be one central "root logger". All other logger objects (such as a logger attached to a block) will be childs of the root logger - Every child logger can filter log messages by severity. Do we also want to filter in the root logger, or do we just inherit the default log level from it? That's up for debate. - The root logger will implement a thread-safe multi-writer queue for log infos. It will serially handle the actual logging, which means the logging is not fully synchronous -- but it also means that the publishing of the log messages is not handled by the thread that is producing it, which is good for performance. - We will most likely start with the dumbest form of such a queue, maybe a std::queue with a mutex and a notification variable. We can optimize later, if need be. Lockfree data structures tend to be worse for best-case and better for worst-case latency, which probably makes them less suitable but who knows. - The root logger will be created at runtime in a singleton pattern. Andrej pointed out that might cause trouble at destruction time, so we should think about that. Maybe only child loggers share a reference to the root logger, and the singleton factory only stores a weak pointer to it. It runs a separate thread to dispatch log messages to the various logging back-ends. On top of that, here's a couple of suggestions from myself, which are the least fixed out of all of these bullet points: - We could further simplify the block logging interface by adding an API call to the block itself, rather than exposing the logger object. This guarantees that the source ID for block log messages is always correct. - I want (optional) colours on the stderr output :) Cheers, Martin On Fri, Dec 28, 2018 at 4:49 PM Michael Dickens wrote: > I'll second what Jared wrote, and add the following: I'd like -any- > logging GR uses to provide protected printing, such that logged messages do > not interleave. [Obviously, unlogged messages might interleave / intermix; > all depends on the actual printing interface.] Trying to parse such > interleaved messages is a total PITA. I've personally gone so far as to > implement a mutex interface via SWIG from C++ to Python, to allow logged > messages in both to not interleave. Ideally, all of stdout and stderr would > be redirected to the logging interface, regardless of language source. > Log4Cpp provides the printing blocking (non-interleaving) functionality, > though it doesn't coerce both C++ and Python into the same printing > interface (at least in my testing). - MLD > > On Fri, Dec 28, 2018, at 7:08 PM, U L wrote: > > I think many users, myself included, use logs for debugging. I would > hesitate to have the log messages propagate through the very system that > you are trying to debug with said messages. I would rather the logs be > emitted from the system under test as simply and promptly as possible. I > think that hierarchical organization within the logs should be handled by > log metadata. It's not clear that one would want to impose the same naming > hierarchy on logs as you do your flowgraph, but one could do so if desired. > > Jared. > > > ___ > Discuss-gnuradio mailing list > Discuss-gnuradio@gnu.org > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio > ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org
Re: [Discuss-gnuradio] What does complexity in Show Flowgraph Complexity in GRC means?
It's mostly just a joke. It has to do with one of the GNU Radio faithful, Balint Seeber. He was/is known for incredibly complex flow graphs. So a ubal is a "micro Balint". Ron On 1/6/19 04:57, Ignatius Rivaldi wrote: When I enabled Show Flowgraph Complexity in GRC, there is a small grey box below the Options box with the writing Complexity: 564ubal. What does that mean? What is ubal? Is higher slower? How high can I go? Where is the documentation for that? I've tried googling 'gnuradio flowgraph complexity'. 'gnuradio show flowgraph complexity' and 'gnuradio ubal' and there is no useful answers from all of those terms Thanks ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
[Discuss-gnuradio] What does complexity in Show Flowgraph Complexity in GRC means?
When I enabled Show Flowgraph Complexity in GRC, there is a small grey box below the Options box with the writing Complexity: 564ubal. What does that mean? What is ubal? Is higher slower? How high can I go? Where is the documentation for that? I've tried googling 'gnuradio flowgraph complexity'. 'gnuradio show flowgraph complexity' and 'gnuradio ubal' and there is no useful answers from all of those terms Thanks ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] std::bad_alloc in Reed-Solomon decoder
El 06/01/19 a las 10:39, Daniel Estévez escribió: > Hi, > > I am trying to debug a problem that some users have found in the ESEO > decoder in gr-satellites (https://github.com/daniestevez/gr-satellites). > > The problem is that after running for a lot of time, the Reed-Solomon > decoder block throws a std::bad_alloc exception, which gets printed out as > > thread[thread-per-block[13]: ]: std::bad_alloc > > It seems I am able to replicate this on my system. > > My question is how to start debugging this. Is there any way in which I > can try to find what in the decode_rs_general block is throwing the > std::bad_alloc exception? > > This decode_rs_general block uses Phil Karn's libfec and it doesn't do > any memory allocation, except on initialization, so I find it weird that > it's giving a std::bad_alloc. See the source of the block here: > > https://github.com/daniestevez/gr-satellites/blob/master/lib/decode_rs_general_impl.cc Hi all, I've just found the problem. The size of the PDU received in the RS decoder wasn't checked for minimum size. If the PDU was shorter than the number of parity check bytes, then pmt::init_u8vector was called with a negative length in https://github.com/daniestevez/gr-satellites/blob/master/lib/decode_rs_general_impl.cc#L113 Best regards, Daniel. ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] HELP with "packet header parser"
hi
i does not work because the packet header parser expect an
"packet_header_default" object instead of a "header_format_base" object
bob
Le dim. 6 janv. 2019 à 02:50, Cinaed Simson a
écrit :
> On 1/5/19 2:26 PM, Alban Meffre wrote:
> > ok sorry my mistake i got "packet_header_default" and
> > "header_format_default" mixed up
> > bob
>
> The error message at the bottom indicates the wrong number arguments.
>
> digital.header_format_default('1101001110010001', 0)
>
> Try
>
> digital.header_format_default('1101001110010001', 0, bps)
>
> where bps is the number of bits per symbol for your problem.
>
> See
>
>Default Header Format Obj.
>
> in the grc.
>
> -- Cinaed
>
>
>
>
> >
> > Le sam. 5 janv. 2019 à 22:13, Alban Meffre > > a écrit :
> >
> > hi all
> >
> > i try to figure out how to use the "packet header parser" block
> > as a formatter i use
> > "digital.header_format_default('1101001110010001', 0)"
> >
> > when i launch i get the error
> > "
> > Traceback (most recent call last):
> > File "/Users/alban/gnuradio/top_block.py", line 146, in
> > main()
> > File "/Users/alban/gnuradio/top_block.py", line 134, in main
> > tb = top_block_cls()
> > File "/Users/alban/gnuradio/top_block.py", line 71, in __init__
> > self.digital_packet_headerparser_b_0 =
> > digital.packet_headerparser_b(hdr_format_dec_0)
> > File
> >
>
> "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gnuradio/digital/digital_swig2.py",
> > line 2651, in make
> > return _digital_swig2.packet_headerparser_b_make(*args)
> > NotImplementedError: Wrong number or type of arguments for
> > overloaded function 'packet_headerparser_b_make'.
> > Possible C/C++ prototypes are:
> >
> >
>
> gr::digital::packet_headerparser_b::make(gr::digital::packet_header_default::sptr
> > const &)
> > gr::digital::packet_headerparser_b::make(long,std::string const
> &)
> > "
> >
> >
> > in the "rx_ofdm" example supplied with gnuradio, the "packet header
> > parser" use the formatter "header_formatter.base()"
> >
> > i do not find any documentation about the base() method
> > as you can see the doxygen is empty
> >
> https://www.gnuradio.org/doc/doxygen/classgr_1_1digital_1_1header__format__base.html#a5a7b2939707146f2b28d7e91e04103c2
> >
> > if i add ".base()" to the formatter in my top_block i get an error
> too
> >
> > please someone can explain me hox to use this block ?
> > thank you all
> > --
> > Alban MEFFRE F4GSW
> >
> >
> >
> >
> > --
> > Alban MEFFRE F4GSW
> >
> >
> >
> > ___
> > Discuss-gnuradio mailing list
> > Discuss-gnuradio@gnu.org
> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> >
>
>
--
Alban MEFFRE F4GSW
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
[Discuss-gnuradio] std::bad_alloc in Reed-Solomon decoder
Hi, I am trying to debug a problem that some users have found in the ESEO decoder in gr-satellites (https://github.com/daniestevez/gr-satellites). The problem is that after running for a lot of time, the Reed-Solomon decoder block throws a std::bad_alloc exception, which gets printed out as thread[thread-per-block[13]: ]: std::bad_alloc It seems I am able to replicate this on my system. My question is how to start debugging this. Is there any way in which I can try to find what in the decode_rs_general block is throwing the std::bad_alloc exception? This decode_rs_general block uses Phil Karn's libfec and it doesn't do any memory allocation, except on initialization, so I find it weird that it's giving a std::bad_alloc. See the source of the block here: https://github.com/daniestevez/gr-satellites/blob/master/lib/decode_rs_general_impl.cc Best regards, Daniel. ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
