My opinion: I think the option-based annotations are the better way to go for this sort of definition of behavior. Additionally, I agree RPCs could have been just as easily expressed this way and potentially they would have been if they were introduced today.
On Thu, Apr 9, 2020 at 4:30 AM Alex Barlow <[email protected]> wrote: > Yes I thought about this and prototyped it out a little bit last night. > > But that’s true of RPC too, it could be done with options, but it has > special status as its own top level semantic which is really nice. > > Hence my proposal. I am considering what you’ve done with options as our > current statedgy going forward for sure, one issue is having to include > that definition everywhere but we can manage that. > > Thanks for the interest > > > On 9 April 2020 at 12:27:14, Alex Van Boxel ([email protected]) wrote: > > If I understand you correctly you want to generate Go, Js and Typescript > code out of your Protobuf DSL. That sounds like a valid approach, but you > don't need to extend the proto DSL from that, you can do that through > message options. > > We use options extensively, although not for generating code. You can > create a protoc plugin for your languages that act on the annotations and > generate code that is specific for the message. > > If you have conventions for your topic, even better you can bake that > logic into the plugin. Example having: > > message Orders { > option (your.option.package.pubsub) = ["Created", "Updated"] > string id = 1; > } > > could generate a client with the verbs "Created" and "Updated" accepting > the Protobuf Order. I don't see the need to extend the protobuf DSL for > this, once you understand options the usages as limitless. > > We don't use options for this, but we use it a-lot in our Big Data > pipelines. > > _/ > _/ Alex Van Boxel > > > On Thu, Apr 9, 2020 at 12:48 PM Alex Barlow <[email protected]> wrote: > >> Hi all, >> >> I'm a big user of protobuf in several roles now and a common theme is >> recreating the publish and subscribe pattern as well as RPC with >> protobuffers. In particular our use case at the moment is publish protobuf >> messages to Google Pub Sub, but have also used/experimented with >> Kafka/Nats/SQS. >> >> The benefits are of course the speed of parsing, typings and forward >> compatibility, but one large downside is the ability to accidentally >> subscribe to the wrong type on a topic or publish the wrong type, at the >> moment, the consumer of a topic provides the type and we reflect that type >> when consuming, for example. >> >> This can look like the following (in Go)... >> >> c.On(pubsub.HandlerOptions{ >> Topic: "name_of_topic", >> Name: "update_order_in_bigquery", >> Handler: func(ctx context.Context, o *order.Order, _ *pubsub.Msg) error { >> return publishToBQ(o) >> }, >> }) >> >> As stated before, if the topic is incorrectly named or the type doesn't >> match the topic, subscribing fails. Worse so in publishing you can publish >> one type to a channel of other types. This message will never be processed. >> >> Ideally we'd like to type both the publisher and subscribers and generate >> the code, much like gRPC and the service definition in protobuf. >> >> I propose something like the following.. >> >> message Orders { >> string id = 1; >> } >> >> service Orders { >> rpc Get(GetRequest) returns (GetResponse); >> rpc Create(CreateRequest) returns (CreateResponse); >> rpc Update(UpdateRequest) returns (UpdateResponse); >> rpc List(ListRequest) returns (ListResponse); >> rpc Subscribe(SubscribeRequest) returns (stream SubscribeResponse) {}; >> } >> >> publisher Orders { >> topic Created(Order); >> topic Updated(Order); >> } >> >> Any thoughts of stories of how other are handling this would be great! >> >> Happy to help contribute too, especially in Go, JS and Typescript domains >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Protocol Buffers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/protobuf/9d25655b-0f1e-4e04-81f9-1925486d93e2%40googlegroups.com >> <https://groups.google.com/d/msgid/protobuf/9d25655b-0f1e-4e04-81f9-1925486d93e2%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/protobuf/CABatEtQMm6gabX9TT0Wgu1NeL4e1Mqo4eJkbVVpmKO-WDUrxyg%40mail.gmail.com > <https://groups.google.com/d/msgid/protobuf/CABatEtQMm6gabX9TT0Wgu1NeL4e1Mqo4eJkbVVpmKO-WDUrxyg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/CAD7-yfvjYbTT-PX1ZEVc20ghT7YNvyRNrz20umpxsiMXsuZ-%2BA%40mail.gmail.com.
