So far from reading the ZMQ examples I have the situation of a shared resource (ring-buffer) by several different workers which work on the buffers. The worker threads are dynamic in the sense that they may be paused and started at runtime, so I need a controller also. My first basic approach would be a single broker controlling the ring buffer state to which clients connect to, request buffers and submit finished buffers. I think a REQ-DEALER-ROUTER-REP chain should be able to handle this. I will have to add a controller somehow too.
On Thu, Jan 17, 2013 at 9:39 AM, A. Mark <[email protected]> wrote: > Thanks Steve, > > > The bottom line in my situation is this, I have a ringbuffer and several > threads with serial dependencies on each other. It's an MPMC situation. My > main goal is minimizing wait and minimizing CPU cost of scheduling threads > at the same time. In my current implementation - which as I mentioned is a > bit of a nightmare with many complicated rules - I use a single > lock/unlock interface. The only relatively "easy" way I found to make it > deadlock-free is to use timeout's on locks and broadcasts once in a while. > It's a messy business even though it does work. The worst part is that it > doesn't scale well since the timeouts and wake-ups consume more and more > CPU. These are wasted wake ups due to timeouts and broadcasts....So here I > am looking into ZMQ and FF. I feel like if ZMQ can scale better I would > rather pay a bit of penalty in performance (I mean slower thread > switching), if it also less "wasteful" for unnecessary wake-ups. I'm only > learning ZMQ and no very little about FF. > > Interesting article, I wish I could use it... > > On Thu, Jan 17, 2013 at 6:04 AM, Steven McCoy <[email protected]>wrote: > >> On 17 January 2013 01:35, A. Mark <[email protected]> wrote: >> >>> I'm needing to implement some low level MPMC queues. I've been rolling >>> my own using pthreads, and so far I've been pulling my hair out in the >>> process also... My code is strictly C at this point and I'd like to keep it >>> that way. I've looked into FF (FastFlow) and find it's rather technical for >>> low level so far. Is there any advantages over using ZMQ for MPMC queues? >>> What would be the pros and cons of using ZMQ over FF for MPMC? It seems to >>> me so far that ZMQ is very well suited for MPMC solutions. >>> >> >> Interesting, the technology paradigm is effectively the same but ZeroMQ >> wraps in an easy to learn and use interface. >> >> Of note from this page, >> >> ... queues, which require neither locks nor atomic operations. SPMC, >>> MPSC, and MPMC queues are realized by using only SPSC queues and an arbiter >>> thread, >> >> >> http://calvados.di.unipi.it/dokuwiki/doku.php?id=ffnamespace:architecture >> >> >> On one hand you have full memory bus speed queues for single-provider >> single-consumer, but on the other you have a broker thread with implied >> synchronisation overheads. >> >> There's a paper to read, that discusses queue implementations without >> atomic primitives: >> >> >> http://calvados.di.unipi.it/dokuwiki/lib/tpl/torquati/paper_files/TR-10-20.pdf >> >> -- >> Steve-o >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
