On Tue, Apr 3, 2018 at 6:47 AM, Kristoffer Sjögren <sto...@gmail.com> wrote:
> Todd: Sending the location for the response in the request was the first > idea that came to mind. But as clients in this case would be very short > lived processes i'm trying to preallocate/setup as much as possible during > server startup. > With a reply-to that contains a file name, the file would be created by the client. So, it should keep latency low for allocation. And would be no allocation on the server.... if that is the side you want to make sure is not held up with allocation. However, it does have both sides mapping the region. An alternative to a reply-to via filename is a single response buffer area. As clients "connect", they atomically increment a counter. That counter (mod the size of the area) dictates their response slot. So, in that case, everything but the client mapping is pre-allocated. It just means you need a section of memory and you have to handle collisions of 2 clients grabbing the same mapping (same mod value). > > Is it feasible to use a shared aeron_mpsc_rb for incoming client requests > and use the aeron_broadcast_transmitter for (async) responses? Where > server indicate the recipient by means of a clientId message header > (other already connected client would ignore this message). Or is the > aeron_broadcast_transmitter best used for protocol/control events without > much payload? I guess the server could also broadcast the location of a > preallocated single client aeron_spsc_rb for larger payloads in the reply? > Exactly! aeron_mpsc_rb + aeron_broadcast is exactly what the Aeron driver does. Everything you said is exactly correct. The responses are small. And some are async in nature. Such as image available/unavailable. A clientId (as well as command correlation Ids) are an atomic value via the aeron_mpsc_rb interface. aeron_mpsc_rb_next_correlation_id to be precise. It's an atomic counter that can be used to generate a unique id for the driver and all clients (as long as they all use it). Aeron doesn't have any large responses. But if it did, it would do exactly as you mention, send the filename of the ring buffer. This is what it effectively does with the logbuffers for publications and subscriptions. > > > > > > On Mon, Apr 2, 2018 at 7:55 PM, Todd Montgomery <tm...@nard.net> wrote: > >> Aeron itself uses a Broadcast scheme for request/response with clients. >> All clients listen to the Broadcast for responses as well as async events. >> A lot of times a pure request/response isn't the right semantic. There is >> often the need for async events and multi-client information. >> >> But if you need to do client/server mechanism with single client response >> buffers, the easiest way is to have the client send the location for the >> response on its request (Reply-To style). Easily done with files, BTW. >> >> On Sun, Apr 1, 2018 at 1:44 PM, Kristoffer Sjögren <sto...@gmail.com> >> wrote: >> >>> What would a multi-client/server protocol look like for two-way session >>> communication >>> using shared memory ring buffers? Especially during setup where both >>> processes needs to agree on where the in/out ring buffers reside in memory. >>> I'm guessing the initial mmap could be expensive? >>> >>> On Fri, Mar 30, 2018 at 6:40 PM, Roman Leventov <leventov...@gmail.com> >>> wrote: >>> >>>> Right, 8 bytes, not 16, I've misread https://github.com/rea >>>> l-logic/agrona/blob/master/agrona/src/main/java/org/agrona/c >>>> oncurrent/ringbuffer/RecordDescriptor.java. Thanks for note. >>>> >>>> On Fri, 30 Mar 2018, 19:16 Martin Thompson, <mjpt...@gmail.com> wrote: >>>> >>>>> Aeron IPC is more functional than a plain queue but I get your point >>>>> that it is not a drop in replacement. >>>>> >>>>> The ring buffers are typical queue based semantics and use an 8 byte >>>>> header in Agrona and Aeron. 4 bytes for message length and 4 bytes for >>>>> message type to give some flexibility. Fixed format messages can optimise >>>>> well but have limited applicability. >>>>> >>>>> On Friday, 30 March 2018 16:36:09 UTC+1, Roman Leventov wrote: >>>>>> >>>>>> Martin, thanks a lot! >>>>>> >>>>>> I thought about Aeron IPC, but as far as I understand it maps to the >>>>>> queue model only when there is a single producer and a single consumer. >>>>>> Also it felt a little too heavyweight for small fixed-sized messages. >>>>>> Generally Aeron's Data frames have 32-byte headers. RingBuffers have only >>>>>> 16-byte headers, and it looks like it could be harmlessly reduced down >>>>>> to 8 >>>>>> or even 0 for e. g. fixed format 32-byte messages. >>>>>> >>>>>> There are implementations of FIFO ring buffers for Java and C++ used >>>>>>> in Aeron for doing exactly this. >>>>>>> >>>>>>> https://github.com/real-logic/aeron/tree/master/aeron-client >>>>>>> /src/main/cpp/concurrent >>>>>>> >>>>>>> https://github.com/real-logic/agrona/tree/master/agrona/src/ >>>>>>> main/java/org/agrona/concurrent >>>>>>> >>>>>>> You could also use Aeron IPC. >>>>>>> >>>>>>> On Friday, 30 March 2018 09:55:23 UTC+1, Roman Leventov wrote: >>>>>>>> >>>>>>>> I think about the possibility of building an asynchronous >>>>>>>> application with back pressure where some upstream operators are in >>>>>>>> Java >>>>>>>> and some downstream ones are in C++. For this purpose, some queues >>>>>>>> would be >>>>>>>> needed to pass the data between Java and C++ layers. It seems that >>>>>>>> porting >>>>>>>> JCTools's bounded array queues to off-heap should be doable, but I >>>>>>>> couldn't >>>>>>>> find existing prototypes or discussions of such thing so maybe I >>>>>>>> overlook >>>>>>>> some inherent complications with this idea. >>>>>>>> >>>>>>>> Did anybody think about something like this or has implemented in >>>>>>>> proprietary systems? >>>>>>>> >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "mechanical-sympathy" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to mechanical-sympathy+unsubscr...@googlegroups.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "mechanical-sympathy" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to mechanical-sympathy+unsubscr...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "mechanical-sympathy" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to mechanical-sympathy+unsubscr...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "mechanical-sympathy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to mechanical-sympathy+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "mechanical-sympathy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to mechanical-sympathy+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group. To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
