Re: RPC Design Structure
Thank you Kenton & Shane - it must have been a slow brain day for me yesterday :-) I confused the underlying implementation with what PB provides. The RPC system doesn't know or care how data is transferred. On Nov 20, 5:56 pm, Shane Green <[EMAIL PROTECTED]> wrote: > It sounds like the system in question has a single "public" service with > delegates calls to back-end services, distributed across machines > available to it. It seems as though the public service provides an > interface which is essentially a composition of the interfaces provided > by the services available to it. That's is exactly. > The process of accepting requests for methods it provides on behalf of > these back-end services, which it handles by delegating to the > appropriate back-end service, is essentially multiplexing multiple > services from a single network location. > > If this is at all accurate, then I believe that the API exported by the > public service should be built, dynamically, based on the set of > interfaces available to it. Taking this approach would allow each > service to define its own interface. While all the functionality > available on the network can be exposed as though it were all provided > by a single entity, assuming there are no naming conflicts ;-) That sounds like what I should be doing. I was just making it hard for myself by thinking it was more complicated than it is. > Or maybe I'm completely confused about the setup. No, I think you've got it right. Thanks, Jeff --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: RPC Design Structure
Thanks for the breakdown, that's very helpful. I had some trouble finding details about how the PB RPC terminology mapped to what I'm familiar with. It sounds like the system in question has a single "public" service with delegates calls to back-end services, distributed across machines available to it. It seems as though the public service provides an interface which is essentially a composition of the interfaces provided by the services available to it. The process of accepting requests for methods it provides on behalf of these back-end services, which it handles by delegating to the appropriate back-end service, is essentially multiplexing multiple services from a single network location. If this is at all accurate, then I believe that the API exported by the public service should be built, dynamically, based on the set of interfaces available to it. Taking this approach would allow each service to define its own interface. While all the functionality available on the network can be exposed as though it were all provided by a single entity, assuming there are no naming conflicts ;-) Or maybe I'm completely confused about the setup. Best regards, Shane On Wed, 2008-11-19 at 22:37 -0800, Kenton Varda wrote: > The design of an RPC system is a large topic and there are lots of > different ways you could do it. The RPC interfaces provided by > Protocol Buffers are meant to provide the minimum support necessary to > allow protoc to generate type-safe service stubs. How you want to > implement them is up to you. > > > This is described in the docs, but to summarize, the basic classes > are: > > > Service: An object that receives messages, possibly from remote > clients. protoc generates an interface corresponding to each service > in a .proto file. These interfaces are implemented by the server > application. > > > RpcChannel: Represents an abstract tunnel to a single service, > allowing you to send messages just to that service. This is an > abstract interface which should be implemented by the RPC library. > > > > RpcController: Manages state related to a single remote procedure call > -- that is, a single message sent to the server, and its corresponding > response. This is an abstract interface which should be implemented > by the RPC library. > > > > Stub: A fake implementation of a service interface which just forwards > messages to an RpcChannel. This makes the service appear to be a > local object when it is not. protoc automatically generates a stub > class for every service type. > > > > > Note that you could easily have multiple RpcChannels that share a > single TCP connection and lead to multiple service objects running on > a single server. The interfaces are designed to put as few > restrictions on implementations as possible. > > > On Wed, Nov 19, 2008 at 9:52 PM, codeazure <[EMAIL PROTECTED]> > wrote: > > OK, now you've confused me :-) > > I don't understand the exact relationship between all these > classes, > which is why I'm asking the question. If I want to build an > application where I have a number of services that share a > single TCP > port, what organisation do I need to use? > > You mention multiplexing services - what does that mean for a > client > application using the connection? > > A UML:diagram (or similar) showing the relationship between > controllers, channels & services would really aid my > understanding of > how this system would operate. Perhaps these terms are in > common usage > in other RPC systems, but because I haven't used any, I'm > uncertain > about what these entities do. I've read the documentation > several > times, but the overview of how it works hasn't clicked. > > Thanks, > Jeff > > On Nov 20, 12:54 pm, Kenton Varda <[EMAIL PROTECTED]> wrote: > > RpcController objects are per-request, not per-server or > per-service. For > > every RPC request you make, you should have another > RpcController object > > (though you can reuse an object by calling Clear() as long > as you aren't > > making two requests at once). > > RpcChannel objects are per-service. Is that what you were > thinking of? A > > single RpcChannel represents a connection to a single > Service. However, > > there's nothing stopping you from multiplexing multiple > RpcChannels across a > > single TCP connection, or creating a protocol that allows > you to choose > > between multiple services exported by a server when > constructing an > > RpcChannel. > > > > > > > > > > --~--~-~--~~~---~--~~ Yo
Re: RPC Design Structure
The design of an RPC system is a large topic and there are lots of different ways you could do it. The RPC interfaces provided by Protocol Buffers are meant to provide the minimum support necessary to allow protoc to generate type-safe service stubs. How you want to implement them is up to you. This is described in the docs, but to summarize, the basic classes are: Service: An object that receives messages, possibly from remote clients. protoc generates an interface corresponding to each service in a .proto file. These interfaces are implemented by the server application. RpcChannel: Represents an abstract tunnel to a single service, allowing you to send messages just to that service. This is an abstract interface which should be implemented by the RPC library. RpcController: Manages state related to a single remote procedure call -- that is, a single message sent to the server, and its corresponding response. This is an abstract interface which should be implemented by the RPC library. Stub: A fake implementation of a service interface which just forwards messages to an RpcChannel. This makes the service appear to be a local object when it is not. protoc automatically generates a stub class for every service type. Note that you could easily have multiple RpcChannels that share a single TCP connection and lead to multiple service objects running on a single server. The interfaces are designed to put as few restrictions on implementations as possible. On Wed, Nov 19, 2008 at 9:52 PM, codeazure <[EMAIL PROTECTED]> wrote: > > OK, now you've confused me :-) > > I don't understand the exact relationship between all these classes, > which is why I'm asking the question. If I want to build an > application where I have a number of services that share a single TCP > port, what organisation do I need to use? > > You mention multiplexing services - what does that mean for a client > application using the connection? > > A UML:diagram (or similar) showing the relationship between > controllers, channels & services would really aid my understanding of > how this system would operate. Perhaps these terms are in common usage > in other RPC systems, but because I haven't used any, I'm uncertain > about what these entities do. I've read the documentation several > times, but the overview of how it works hasn't clicked. > > Thanks, > Jeff > > On Nov 20, 12:54 pm, Kenton Varda <[EMAIL PROTECTED]> wrote: > > RpcController objects are per-request, not per-server or per-service. > For > > every RPC request you make, you should have another RpcController object > > (though you can reuse an object by calling Clear() as long as you aren't > > making two requests at once). > > RpcChannel objects are per-service. Is that what you were thinking of? > A > > single RpcChannel represents a connection to a single Service. However, > > there's nothing stopping you from multiplexing multiple RpcChannels > across a > > single TCP connection, or creating a protocol that allows you to choose > > between multiple services exported by a server when constructing an > > RpcChannel. > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: RPC Design Structure
OK, now you've confused me :-) I don't understand the exact relationship between all these classes, which is why I'm asking the question. If I want to build an application where I have a number of services that share a single TCP port, what organisation do I need to use? You mention multiplexing services - what does that mean for a client application using the connection? A UML:diagram (or similar) showing the relationship between controllers, channels & services would really aid my understanding of how this system would operate. Perhaps these terms are in common usage in other RPC systems, but because I haven't used any, I'm uncertain about what these entities do. I've read the documentation several times, but the overview of how it works hasn't clicked. Thanks, Jeff On Nov 20, 12:54 pm, Kenton Varda <[EMAIL PROTECTED]> wrote: > RpcController objects are per-request, not per-server or per-service. For > every RPC request you make, you should have another RpcController object > (though you can reuse an object by calling Clear() as long as you aren't > making two requests at once). > RpcChannel objects are per-service. Is that what you were thinking of? A > single RpcChannel represents a connection to a single Service. However, > there's nothing stopping you from multiplexing multiple RpcChannels across a > single TCP connection, or creating a protocol that allows you to choose > between multiple services exported by a server when constructing an > RpcChannel. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: RPC Design Structure
RpcController objects are per-request, not per-server or per-service. For every RPC request you make, you should have another RpcController object (though you can reuse an object by calling Clear() as long as you aren't making two requests at once). RpcChannel objects are per-service. Is that what you were thinking of? A single RpcChannel represents a connection to a single Service. However, there's nothing stopping you from multiplexing multiple RpcChannels across a single TCP connection, or creating a protocol that allows you to choose between multiple services exported by a server when constructing an RpcChannel. On Wed, Nov 19, 2008 at 5:06 PM, codeazure <[EMAIL PROTECTED]> wrote: > > On Nov 20, 7:58 am, Kenton Varda <[EMAIL PROTECTED]> wrote: > > I'm not sure I understand. There's nothing stopping you from spreading > your > > definitions out among multiple .proto files which import each other, and > > there's nothing stopping you from exporting multiple services from a > single > > server. You'll need to design a protocol that allows it, but protocol > > buffers doesn't do anything to prevent it. Can you be more specific > about > > the problem you're facing, maybe giving an example? > > I can have multiple services per server? That's what I wanted to know. > I could not any place in the documentation where it talked about > whether Service has a 1:1 or Many:1 relationship with RpcController. > It seemed like it was 1:1 from all the examples I had seen, hence my > confusion. Next time you're updating the docs in this area, it might > be worth changing the example to show 2 Services so it's clear to > others. > > I was also thinking of services running on ports on Linux systems. In > this situation, it is a 1:1 relationship between services and ports. I > know it's a different situation, but my thinking got stuck on the term > service. > > If I can define as many services as I like and attach them all to the > same RPC Controller, then that will answer my question & give me the > extensible modular design I want. I had thought there had to be an > answer like this because I couldn't imagine Google's internal > applications would want to be bound by such a serious limitation. > > Thanks! > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: RPC Design Structure
On Nov 20, 7:58 am, Kenton Varda <[EMAIL PROTECTED]> wrote: > I'm not sure I understand. There's nothing stopping you from spreading your > definitions out among multiple .proto files which import each other, and > there's nothing stopping you from exporting multiple services from a single > server. You'll need to design a protocol that allows it, but protocol > buffers doesn't do anything to prevent it. Can you be more specific about > the problem you're facing, maybe giving an example? I can have multiple services per server? That's what I wanted to know. I could not any place in the documentation where it talked about whether Service has a 1:1 or Many:1 relationship with RpcController. It seemed like it was 1:1 from all the examples I had seen, hence my confusion. Next time you're updating the docs in this area, it might be worth changing the example to show 2 Services so it's clear to others. I was also thinking of services running on ports on Linux systems. In this situation, it is a 1:1 relationship between services and ports. I know it's a different situation, but my thinking got stuck on the term service. If I can define as many services as I like and attach them all to the same RPC Controller, then that will answer my question & give me the extensible modular design I want. I had thought there had to be an answer like this because I couldn't imagine Google's internal applications would want to be bound by such a serious limitation. Thanks! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: RPC Design Structure
I'm not sure I understand. There's nothing stopping you from spreading your definitions out among multiple .proto files which import each other, and there's nothing stopping you from exporting multiple services from a single server. You'll need to design a protocol that allows it, but protocol buffers doesn't do anything to prevent it. Can you be more specific about the problem you're facing, maybe giving an example? On Wed, Nov 19, 2008 at 12:21 AM, codeazure <[EMAIL PROTECTED]> wrote: > > I've been struggling to work out how to use PB's RPC mechanism within > my application. I don't have a problem with the simple usage of it or > the transport side. It's more how I would scale it up within a larger > modular application. > > My application can run on multiple machines that need to talk to each > other. Think of it as a classic distributed computing problem. > > This application is composed of a number of modules. Some of them will > have one or more functions that need to be called via RPC. Different > builds of the application may use some different modules (for device > drivers, etc). > > I plan to have a TCP server socket that will run at a predefined port > that will provide the connection for one instance of my application to > talk to others of its kind. So each machine can be configured to > connect to one or more others using this port as the transport for RPC > messages. Fairly standard way of doing things I hope. > > My problem is how do I use the RPC interface in PB to bind all the RPC > function calls scattered across my application? The only way I can see > is to have a single proto file that defines everything from every > module in a single place. > > From a design point of view, I find this somewhat ugly as it makes a > mockery of all my fine efforts to design software in a modular way. It > also makes it very tricky if I'm defining the interface for modules > that may not be enabled in a given implementation. > > Is there another approach I'm missing? Can each module have it's own > local RPC section that is somehow combined in the complete application > to present a unified interface? > > From my understanding, there is only a single Service and Controller > defined for a single RPC server. Is this wrong? Can I define a bunch > of Services and bind them to a single port? > > Sorry for the long email - it's a tricky thing to explain. > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
RPC Design Structure
I've been struggling to work out how to use PB's RPC mechanism within my application. I don't have a problem with the simple usage of it or the transport side. It's more how I would scale it up within a larger modular application. My application can run on multiple machines that need to talk to each other. Think of it as a classic distributed computing problem. This application is composed of a number of modules. Some of them will have one or more functions that need to be called via RPC. Different builds of the application may use some different modules (for device drivers, etc). I plan to have a TCP server socket that will run at a predefined port that will provide the connection for one instance of my application to talk to others of its kind. So each machine can be configured to connect to one or more others using this port as the transport for RPC messages. Fairly standard way of doing things I hope. My problem is how do I use the RPC interface in PB to bind all the RPC function calls scattered across my application? The only way I can see is to have a single proto file that defines everything from every module in a single place. >From a design point of view, I find this somewhat ugly as it makes a mockery of all my fine efforts to design software in a modular way. It also makes it very tricky if I'm defining the interface for modules that may not be enabled in a given implementation. Is there another approach I'm missing? Can each module have it's own local RPC section that is somehow combined in the complete application to present a unified interface? >From my understanding, there is only a single Service and Controller defined for a single RPC server. Is this wrong? Can I define a bunch of Services and bind them to a single port? Sorry for the long email - it's a tricky thing to explain. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
