Re: Intra-process queue between Java and C++

On Tue, Apr 3, 2018 at 6:47 AM, Kristoffer Sjögren wrote: > Todd: Sending the location for the response in the request was the first > idea that came to mind. But as clients in this case would be very short > lived processes i'm trying to preallocate/setup as much as possible

Re: Intra-process queue between Java and C++

> > The broadcast buffer is a very unique type of structure. So true! It relies on a nice mechanics similar to StampedLock and is quite ineresting indeed: I have implemented for Agrona a version of it based on the FastFlow-like logic instead to

Re: Intra-process queue between Java and C++

Good points. The Agrona (and Aeron C versions) do have the proper basic logic to avoid blocked queues from failure during appends/offers. It's not simple, but the hard part is done for you. Just need to hook the logic into a duty cycle. The broadcast buffer is a very unique type of structure. It

Re: Intra-process queue between Java and C++

There are a couple of things to be considered: 1. size request and response buffers to avoid deadlocks (there is an interesting topic on this list re it too) -> could be used a broadcast buffer too to avoid it, but it change the semantic by adding a new "failure" state 2.

Re: Intra-process queue between Java and C++

Todd: Sending the location for the response in the request was the first idea that came to mind. But as clients in this case would be very short lived processes i'm trying to preallocate/setup as much as possible during server startup. Is it feasible to use a shared aeron_mpsc_rb for incoming

Re: Intra-process queue between Java and C++

Aeron itself uses a Broadcast scheme for request/response with clients. All clients listen to the Broadcast for responses as well as async events. A lot of times a pure request/response isn't the right semantic. There is often the need for async events and multi-client information. But if you

Re: Intra-process queue between Java and C++

If you can tolerate the overhead of JNI it sounds like zeromq (zeromq.org) might fit the bill: though as far as i know there is no built in back pressure other than a high watermark on queued messages for a socket. On Friday, March 30, 2018 at 9:55:23 AM UTC+1, Roman Leventov wrote: > > I

Re: Intra-process queue between Java and C++

Not much, I think. Anyway, as Martin pointed to OneToOneRingBuffer it's irrelevant now. On Fri, 30 Mar 2018, 19:17 Greg Young, wrote: > Will the 16 bytes save you time? > > On Fri, Mar 30, 2018 at 10:36 PM, Roman Leventov > wrote: > >> Martin,

Re: Intra-process queue between Java and C++

Right, 8 bytes, not 16, I've misread https://github.com/real-logic/agrona/blob/master/agrona/src/main/java/org/agrona/concurrent/ringbuffer/RecordDescriptor.java. Thanks for note. On Fri, 30 Mar 2018, 19:16 Martin Thompson, wrote: > Aeron IPC is more functional than a plain

Re: Intra-process queue between Java and C++

Will the 16 bytes save you time? On Fri, Mar 30, 2018 at 10:36 PM, Roman Leventov wrote: > Martin, thanks a lot! > > I thought about Aeron IPC, but as far as I understand it maps to the queue > model only when there is a single producer and a single consumer. Also it >

Re: Intra-process queue between Java and C++

Aeron IPC is more functional than a plain queue but I get your point that it is not a drop in replacement. The ring buffers are typical queue based semantics and use an 8 byte header in Agrona and Aeron. 4 bytes for message length and 4 bytes for message type to give some flexibility. Fixed

Re: Intra-process queue between Java and C++

Martin, thanks a lot! I thought about Aeron IPC, but as far as I understand it maps to the queue model only when there is a single producer and a single consumer. Also it felt a little too heavyweight for small fixed-sized messages. Generally Aeron's Data frames have 32-byte headers. RingBuffers