On Thu, Dec 10, 2009 at 3:02 PM, Mikhail Opletayev <opleta...@gmail.com>wrote:

> Will there be a way to directly connect to the service or a client
> must go thru the discovery service each time a connection is open?
>

Why build this into the protocol when the discovery service is just as good?
 Note that the protocol supports sending a request to the returned service
before actually waiting for it to be returned.  In other words, a
conversation might look like:

client:  "Open service Foo."
client:  "When request 1 is complete, call method Bar() on the result."
server:  "Foo opened successfully.  Here is a reference."
server:  "Bar completed.  Here is the response."

By keeping service naming out of the low-level protocol, we keep it nice and
simple, which makes it easier to write a high-quality implementation.

I also intend to provide a standard "discovery service" interface, but it
will be a layer that sits on top of the raw protocol implementation, rather
than being built into it.  Applications will be able to use the raw protocol
without using the standard discovery service if desired.


> > What do you mean?  The global "Stream" message exists only to define the
> > protocol.  It is not actually used at runtime -- individual messages are
> > read from the stream one at a time.
>
> From pbcap.proto it looked to me as all requests and responses must be
> wrapped inside a Stream message. I guess I'll need to take a look at
> the implementation before I ask more questions, I only had time to
> take a quick peak earlier today.
>

Maybe what you don't like is the fact that the app-specific request message
is embedded as "bytes" in pbcap.CallRequest?  Currently this requires a
reduntant copy of all those bytes.  However, I was thinking of adding an
optimization to the protobuf implementation to avoid this -- we could add a
way to construct a new ByteString that is a substring of some other
ByteString in constant time (with no copies).  Then, parsing "bytes" fields
-- when the original input is a ByteString -- would not require a copy,
making things efficient.


>
> P.S. I hope my questions don't derail this thread too much as the
> original question was about HTTP RPC.
>
> On Dec 10, 4:42 pm, Kenton Varda <ken...@google.com> wrote:
> > On Thu, Dec 10, 2009 at 2:37 PM, Mikhail Opletayev <opleta...@gmail.com
> >wrote:
> >
> > > Interesting. Essentially a discovery service for protobuf RPC.
> >
> > > I am not quite sure what you mean by "pointers to other services". Is
> > > it going to reference them by name or a more complex structure
> > > containing full endpoint information?
> >
> > Currently it references them by an ID number that is tied to the
> particular
> > connection.  So, each time a new service object is returned on the
> > connection, a new ID number is assigned to it.
> >
> > > Also, is it going to be an extension to pbcap or something completely
> > > new?
> >
> > Not an extension -- this *is* pbcap.  It supports this already.
> >
> > (Note that I'm planning to change the name to "Captain Proto", aka
> > capnproto, to avoid the confusion with pcap.)
> >
> > > The reason I am asking is because some patterns in pbcap (such as
> > > wrapping up everything into a global Stream message) are rather
> > > questionable and not without consequences.
> >
> > What do you mean?  The global "Stream" message exists only to define the
> > protocol.  It is not actually used at runtime -- individual messages are
> > read from the stream one at a time.
> >
> >
> >
> > > Regards,
> >
> > > On Dec 10, 4:22 pm, Kenton Varda <ken...@google.com> wrote:
> > > > On Thu, Dec 10, 2009 at 8:13 AM, Mikhail Opletayev <
> opleta...@gmail.com
> > > >wrote:
> >
> > > > > It's great news that you working on a standard way to communicate
> > > > > between Protocol Buffers implementations!
> >
> > > > > > You don't need to send the service name at all.  The server
> should
> > > > > already
> > > > > > know what kind of service it is exporting.
> >
> > > > > I think its handy to export several services from the same end
> point,
> > > > > especially if you are running RPC over something else than HTTP. If
> > > > > you were to run Protocol Buffers RPC over plain sockets you'd
> probably
> > > > > want to publish a bunch of services on the same port.
> >
> > > > This is exactly my point.  If you use the service type name to
> identify
> > > the
> > > > service, then you can only export one service of each type.  Instead,
> > > some
> > > > other name -- having nothing to do with the type name -- should be
> used
> > > to
> > > > identify the service.
> >
> > > > Actually, the implementation I'm working on doesn't even identify
> > > services
> > > > by names.  Instead, when you first connect on a port, you connect to
> the
> > > > "default service" for that port.  However, the default service can
> send
> > > back
> > > > pointers to other services in RPC responses.  So, the default service
> may
> > > > have a method which looks up other services by name, but this is up
> to
> > > the
> > > > application.
> >
> > > > > In Dataflow implementation we use one field (method) but we require
> > > > > the method name to be in "serviceName.methodName" format.
> >
> > > > > For the same reason we decided against using HTTP headers to
> transfer
> > > > > the RPC metadata as it binds you to the transport protocol. That's
> why
> > > > > send content length and header length as the first 2 values in the
> > > > > coded stream, then the header message, then the actual data
> message:
> >
> > > > >http://www.dataflow-software.com/docs/pbuf-rpc.html
> >
> > > > > On Dec 10, 3:15 am, Kenton Varda <ken...@google.com> wrote:
> > > > > > 2009/12/10 Romain François <francoisrom...@free.fr>
> >
> > > > > > > On 12/09/2009 09:12 PM, Kenton Varda wrote:
> >
> > > > > > >> Coincidentally, last weekend I started working on an open
> source
> > > > > > >> protobuf-based RPC system.  Currently I am defining a
> socket-level
> > > > > > >> protocol, but I also intend to support an HTTP-level protocol
> with
> > > > > > >> optional JSON encoding to allow calls from Javascript.  I
> stuck
> > > some
> > > > > > >> totally undocumented code here:
> >
> > > > > > > Thanks. My intention of having it over http is that it can
> > > communicate
> > > > > with
> > > > > > > other languages. I'd be good if we can synchronize our
> protocols.
> >
> > > > > > > I need to make some changes based on what you said on another
> > > thread,
> > > > > and
> > > > > > > then I'll make my java basic server code available.
> >
> > > > > > >  http://pbcap.googlecode.com
> >
> > > > > > >> But some coworkers pointed out that the name is confusingly
> > > similar to
> > > > > > >> "pcap", so I'm planning to change it.
> >
> > > > > > >> Currently this is not an official Google project; I'm working
> on
> > > it in
> > > > > > >> my spare time.
> >
> > > > > > >> 2009/12/9 Romain François <francoisrom...@free.fr
> > > > > > >> <mailto:francoisrom...@free.fr>>
> >
> > > > > > >>    A request looks like this :
> >
> > > > > > >>    -----------------------------------------------------
> > > > > > >>    POST /{service full name}/{method name} HTTP/1.0
> >
> > > > > > >> I would recommend against putting the service type name in the
> > > URL.
> > > > > > >>  This makes it impossible to export two services of the same
> type.
> > > > > > >>  Instead, you should allow the application to register
> services
> > > under
> > > > > > >> any name it chooses.
> >
> > > > > > > Fair enough. Maybe adding some protobuf specific headers :
> >
> > > > > > > ProtobufService: {service full name}
> >
> > > > > > You don't need to send the service name at all.  The server
> should
> > > > > already
> > > > > > know what kind of service it is exporting.
> >
> > > > > > > ProtobufMethod: {method full name}
> >
> > > > > > You do need the method name, though.  Inventing new HTTP headers
> > > isn't
> > > > > > usually a good idea as they may confuse proxies and such.
> >
> > > > > > > or encode it as parameters of the url as you said.
> >
> > > > > > >  I'd also suggest making the method name be part of the query,
> > > like:
> >
> > > > > > >>   POST /someservice?method={method name}
> >
> > > > > > >> This may be a matter of taste, but I feel like a service
> object
> > > should
> > > > > > >> be a single HTTP "resource", rather than have each method be a
> > > > > separate
> > > > > > >> resource.
> >
> > > > > > >>    Connection: close
> >
> > > > > > >> Why not allow pipelining?
> >
> > > > > > > this was simpler to do a one shot service call, but indeed why
> not.
> >
> > > > > > >     Content-Length: {length of the serialized message}
> >
> > > > > > >>    {raw bytes of the serialized message}
> > > > > > >>    -----------------------------------------------------
> >
> > > > > > >>    And a successful response looks like this:
> >
> > > > > > >>    -----------------------------------------------------
> > > > > > >>    HTTP/1.1 200 OK
> > > > > > >>    Content-length: {length of the serialized response}
> >
> > > > > > >>    {raw bytes of the serialized response}
> > > > > > >>    -----------------------------------------------------
> >
> > > > > > >>    Since this is very early in this, I wondered if others
> would
> > > have
> > > > > views
> > > > > > >>    on this.
> >
> > > > > > >>    http is quite verbose for sending protobuf message around,
> but
> > > it
> > > > > is
> > > > > > >>    likely to be implemented for a lot of languages.
> >
> > > > > > >>    Regards,
> >
> > > > > > >>    Romain
> >
> > > > > > > --
> > > > > > > Romain Francois
> > > > > > > Professional R Enthusiast
> > > > > > > +33(0) 6 28 91 30 30
> > > > > > >http://romainfrancois.blog.free.fr
> > > > > > > |-http://tr.im/Gq7i:ohloh
> > > > > > > |-http://tr.im/FtUu:newpackage : highlight
> > > > > > > `-http://tr.im/EAD5:LondonRslides
> >
> > > > > --
> >
> > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > "Protocol Buffers" group.
> > > > > To post to this group, send email to proto...@googlegroups.com.
> > > > > To unsubscribe from this group, send email to
> > > > > protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com>
> <protobuf%2bunsubscr...@googlegroups.com<protobuf%252bunsubscr...@googlegroups.com>
> >
> > > <protobuf%2bunsubscr...@googlegroups.com<protobuf%252bunsubscr...@googlegroups.com>
> <protobuf%252bunsubscr...@googlegroups.com<protobuf%25252bunsubscr...@googlegroups.com>
> >
> >
> > > > > .
> > > > > For more options, visit this group at
> > > > >http://groups.google.com/group/protobuf?hl=en.
> >
> > > --
> > > Mikhail Opletayev
> > >http://dataflow-software.com
> >
> > > --
> >
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Protocol Buffers" group.
> > > To post to this group, send email to proto...@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com>
> <protobuf%2bunsubscr...@googlegroups.com<protobuf%252bunsubscr...@googlegroups.com>
> >
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/protobuf?hl=en.
>
> --
>
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to proto...@googlegroups.com.
> To unsubscribe from this group, send email to
> protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/protobuf?hl=en.
>
>
>

--

You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.


Reply via email to