to fulfill the void left by my previous message ;-) here is ConcurrentRequestObserver which eases up developing bi-di streaming methods that dispatch work to other threads but don't need to preserve order of responses:
https://github.com/morgwai/grpc-utils/blob/master/src/main/java/pl/morgwai/base/grpc/utils/ConcurrentRequestObserver.java it handles all the synchronization and manual flow control to maintain the desired concurrency level while also preventing an excessive response messages buffering. This one is gRPC specific as websockets do not have half-closing and manual flow control, so not much left to ease-up there ;-] Cheers! On Friday, April 30, 2021 at 4:10:25 PM UTC+7 Piotr Morgwai Kotarbinski wrote: > please note that this class should only be used if the response messages > order requirement cannot be dropped: if you control a given proto interface > definition, then it's more efficient to add some unique id to request > messages, include it in response messages and send them as soon as they are > produced, so nothing needs to be buffered. > > Cheers! > > On Friday, April 30, 2021 at 8:59:11 AM UTC+7 nitish bhardwaj wrote: > >> >> Thanks for contributing *OrderedConcurrentOutputBuffer. * I totally >> agree, this would be really useful to utilize cores efficiently. >> >> >> On Wednesday, April 28, 2021 at 1:27:46 PM UTC+5:30 [email protected] >> wrote: >> >>> Thanks :) >>> >>> I was surprised actually, because I thought parallel request message >>> processing was a common use-case, both for gRPC and websockets >>> (for example if we have a service that does some single-threaded graphic >>> processing on received images and sends back a modified version of a given >>> image, it would be most efficient to dispatch the processing to a thread >>> pool with a size corresponding to available CPU/GPU cores, right? As >>> processing them sequentially would utilize just 1 core per request stream, >>> so in case of low number of concurrent request streams, we would be >>> underutilizing the cores). >>> >>> Cheers! >>> >>> On Wednesday, April 28, 2021 at 6:52:08 AM UTC+7 Eric Anderson wrote: >>> >>>> Yeah, we don't have anything pre-existing that does something like >>>> that; it gets into the specifics of your use-case. Making something >>>> yourself was appropriate. I will say that the strategy used in >>>> OrderedConcurrentOutputBuffer with the Buckets seems really clean. >>>> >>>> On Thu, Apr 22, 2021 at 9:21 AM Piotr Morgwai Kotarbinski < >>>> [email protected]> wrote: >>>> >>>>> in case someone needs it also, I've written it myself due to lack of >>>>> answers either here and on SO: >>>>> >>>>> https://github.com/morgwai/java-utils/blob/master/src/main/java/pl/morgwai/base/utils/OrderedConcurrentOutputBuffer.java >>>>> feedback is welcome :) >>>>> On Tuesday, April 20, 2021 at 11:09:59 PM UTC+7 Piotr Morgwai >>>>> Kotarbinski wrote: >>>>> >>>>>> Hello >>>>>> i have a stream of messages coming from a websocket or a grpc client. >>>>>> for each message my service produces 0 or more reply messages. by >>>>>> default >>>>>> both websocket endpoints and grpc request observers are guaranteed to be >>>>>> called by maximum 1 thread concurrently, so my replies are sent in the >>>>>> same >>>>>> order as requests. Now I want to dispatch request processing to other >>>>>> threads and process them in parallel, but still keep the order. >>>>>> Therefore, >>>>>> I need some "concurrent ordered response buffer", which will buffer >>>>>> replies >>>>>> to a given request message until processing of previous requests is >>>>>> finished and replies to them are sent (in order they were produced >>>>>> within >>>>>> each "request bucket"). >>>>>> >>>>>> I can develop such class myself, but it seems a common case, so I was >>>>>> wondering if maybe such thing already exists (to not reinvent the >>>>>> wheel). >>>>>> however I could not easily find anything on the web nor get any answer >>>>>> on >>>>>> SO >>>>>> <https://stackoverflow.com/questions/67174565/java-concurrent-ordered-response-buffer> >>>>>> . does anyone knows about something like this? >>>>>> >>>>>> Thanks! >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "grpc.io" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/grpc-io/e7107eed-fa35-4b2e-8d5a-5754e0a37740n%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/grpc-io/e7107eed-fa35-4b2e-8d5a-5754e0a37740n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/ce11d449-db38-494c-b7ea-3c6cf9f7e910n%40googlegroups.com.
