I haven't worked with a message broker myself but I think you might find
that gRPC can work well in that scenario, too. gRPC is not really tied to
web services although that might be a common use case. It supports
different variations
<https://grpc.io/docs/guides/concepts.html#rpc-life-cycle> of streaming
requests and responses, so you're not limited to the typical
send-one-request-and-wait-for-the-response model. You could stream a bunch
of messages to the server without having to wait for a response for each
one. I'm not sure if that necessarily helps with the message broker setup
but I think there's a lot of flexibility there.

On Mon, Jul 2, 2018 at 1:24 PM John Lilley <whee...@gmail.com> wrote:

> Adam,
> Thanks! all good suggestions. These confirm what I'm discovering on my
> own. Simple framing packet containing the message name and bytes. I've
> found the classes/methods for creating a concrete message given its name
> and byte-buffer. So now I'm trying to find a technique for generating both
> server-side and client-side code with request/response methods operating on
> concrete message type.  It seems like gRPC is the only "silver bullet", but
> it is tied to web services.  Meanwhile we're heading down a message-broker
> path, so marrying these two things together is the trick.  LMK if you think
> of something clever or if gRPC can be tricked into using a message-queue
> instead.
> Cheers,
> john
>
>
> On Mon, Jul 2, 2018 at 2:05 PM Adam Cozzette <acozze...@google.com> wrote:
>
>> I realized I missed the fact that in your subject line you mentioned
>> reference counting. In that case it sounds like you might want to use
>> std::shared_ptr instead of std::unique_ptr, but that should also work.
>>
>> Arenas are good for performance but come at the cost of being a little
>> more difficult to work with, so if you just want to make sure the message
>> is destroyed after use in a simple way then I would guess arenas aren't the
>> best option. The simplest thing would be to put your message directly on
>> the stack so that it gets destroyed when you return from the function, but
>> of course that doesn't work if you need the message to outlive the function
>> call. So if putting it on the stack doesn't work for your use case, I would
>> recommend going with std::unique_ptr or std::shared_ptr.
>>
>> About your polymorphism question, I think a good solution would be to
>> create your own class or interface and have that wrap the proto message,
>> since that way your dispatcher could operate on multiple kinds of message
>> without necessarily having to know the concrete message type. This
>> polymorphism could handle  your message->method mapping since you could
>> have a common interface with a separate implementation for each message
>> type. Serialized protocol buffers do not indicate their type or size, so
>> you usually have to set up your own framing to at least indicate the
>> message size. It's common to use a varint size prefix before the message
>> payload, though that's not the only way to do it. To indicate the type of
>> the message, probably the easiest approach is to have a parent message that
>> contains all possible message types inside a oneof
>> <https://developers.google.com/protocol-buffers/docs/proto#oneof> field.
>>
>> After writing all this it occurs to me that you might just want to go
>> with an RPC framework like gRPC <https://grpc.io/>, since that would set
>> up all the infrastructure you need for having service endpoints operating
>> on protocol buffers--have you looked into gRPC by any chance?
>>
>> On Mon, Jul 2, 2018 at 11:19 AM John Lilley <whee...@gmail.com> wrote:
>>
>>>
>>> Thanks Adam!
>>>
>>> I'll look into arena allocation, since all I really want is
>>> destroy-after-use. Will that do it?
>>>
>>> Regarding "polymorphism", what I'm really after is some way to manage
>>> and act upon a group of messages, something like:
>>> 1. Get incoming packet from a queue, deserialize it and create the
>>> message object.
>>> 2. Send the message object (hence the need for a base class) to a
>>> "dispatcher"
>>> 3. Dispatcher invokes the appropriate processing method which creates a
>>> response message.
>>> 4. Serialize the response and send packet on return queue.
>>>
>>> Steps 1 and 3 are the parts that I'm grappling with, both in Java and
>>> C++.  Any tips on
>>> -- The best way to examine the packet and create the correct message?  I
>>> have some freedom to put framing around the packet if necessary.
>>> -- The best way to create and invoke a message->method mapping in Java /
>>> C++?
>>>
>>> Thanks
>>> john
>>>
>>>
>>> On Fri, Jun 29, 2018 at 4:13 PM Adam Cozzette <acozze...@google.com>
>>> wrote:
>>>
>>>> I've found that std::unique_ptr usually works well with messages in
>>>> C++. Probably the only case where things get a little bit complicated is if
>>>> you're using arena allocation
>>>> <https://developers.google.com/protocol-buffers/docs/reference/arenas>,
>>>> since in that case the arena owns the message and you must not call delete
>>>> on it. Within Google we have a special pointer type that keeps track of
>>>> whether a message was arena-allocated or not, and makes sure to call delete
>>>> only if the message was heap-allocated.
>>>>
>>>> About inheritance, there is no way to make all messages inherit from a
>>>> specified class. However, all messages do inherit from
>>>> google::protobuf::MessageLite (and google::protobuf::Message if you're not
>>>> using the lite runtime), so that can be useful if you want to write code
>>>> that operates on arbitrary message types.
>>>>
>>>> On Thu, Jun 28, 2018 at 12:31 PM John Lilley <whee...@gmail.com> wrote:
>>>>
>>>>> I have two (maybe unrelated) questions:
>>>>> -- Is there a recommended way to wrap "smart pointers" around protobuf
>>>>> message objects in C++?
>>>>> -- Is there a way to make all messages inherit from the same base
>>>>> class or interface?  Note I am not trying to make message definitions
>>>>> inherit from each other; I only want the classes generated C++ code to
>>>>> inherit from a class that I control.'
>>>>> Thanks
>>>>> john
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Protocol Buffers" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to protobuf+unsubscr...@googlegroups.com.
>>>>> To post to this group, send email to protobuf@googlegroups.com.
>>>>> Visit this group at https://groups.google.com/group/protobuf.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Protocol Buffers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to protobuf+unsubscr...@googlegroups.com.
>>> To post to this group, send email to protobuf@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/protobuf.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Reply via email to