If I understand you correctly you want to generate Go, Js and Typescript
code out of your Protobuf DSL. That sounds like a valid approach, but you
don't need to extend the proto DSL from that, you can do that through
message options.
We use options extensively, although not for generating code. You can
create a protoc plugin for your languages that act on the annotations and
generate code that is specific for the message.
If you have conventions for your topic, even better you can bake that logic
into the plugin. Example having:
message Orders {
option (your.option.package.pubsub) = ["Created", "Updated"]
string id = 1;
}
could generate a client with the verbs "Created" and "Updated" accepting
the Protobuf Order. I don't see the need to extend the protobuf DSL for
this, once you understand options the usages as limitless.
We don't use options for this, but we use it a-lot in our Big Data
pipelines.
_/
_/ Alex Van Boxel
On Thu, Apr 9, 2020 at 12:48 PM Alex Barlow <[email protected]> wrote:
> Hi all,
>
> I'm a big user of protobuf in several roles now and a common theme is
> recreating the publish and subscribe pattern as well as RPC with
> protobuffers. In particular our use case at the moment is publish protobuf
> messages to Google Pub Sub, but have also used/experimented with
> Kafka/Nats/SQS.
>
> The benefits are of course the speed of parsing, typings and forward
> compatibility, but one large downside is the ability to accidentally
> subscribe to the wrong type on a topic or publish the wrong type, at the
> moment, the consumer of a topic provides the type and we reflect that type
> when consuming, for example.
>
> This can look like the following (in Go)...
>
> c.On(pubsub.HandlerOptions{
> Topic: "name_of_topic",
> Name: "update_order_in_bigquery",
> Handler: func(ctx context.Context, o *order.Order, _ *pubsub.Msg) error {
> return publishToBQ(o)
> },
> })
>
> As stated before, if the topic is incorrectly named or the type doesn't
> match the topic, subscribing fails. Worse so in publishing you can publish
> one type to a channel of other types. This message will never be processed.
>
> Ideally we'd like to type both the publisher and subscribers and generate
> the code, much like gRPC and the service definition in protobuf.
>
> I propose something like the following..
>
> message Orders {
> string id = 1;
> }
>
> service Orders {
> rpc Get(GetRequest) returns (GetResponse);
> rpc Create(CreateRequest) returns (CreateResponse);
> rpc Update(UpdateRequest) returns (UpdateResponse);
> rpc List(ListRequest) returns (ListResponse);
> rpc Subscribe(SubscribeRequest) returns (stream SubscribeResponse) {};
> }
>
> publisher Orders {
> topic Created(Order);
> topic Updated(Order);
> }
>
> Any thoughts of stories of how other are handling this would be great!
>
> Happy to help contribute too, especially in Go, JS and Typescript domains
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/protobuf/9d25655b-0f1e-4e04-81f9-1925486d93e2%40googlegroups.com
> <https://groups.google.com/d/msgid/protobuf/9d25655b-0f1e-4e04-81f9-1925486d93e2%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/protobuf/CALCMntsWDQhcaBKRdbwCjt5TRtnYbCbvHT4foyGjimqFeO7gJg%40mail.gmail.com.
