Responses inline.. > On Wed, Aug 1, 2012 at 11:04 AM, Todd Lipcon <[email protected]> wrote: >> One possibility: >> >> During the IPC handshake, we could send the full version string / >> source checksum. Then, have a client-wide map which caches which >> methods have been found to be supported or not supported for an >> individual version. So, we don't need to maintain the mapping >> ourselves, but we also wouldn't need to do the full retry every time. >>
Yeah this is what I was thinking as the alternate to the current approach of using VersionedProtocol. >> A different idea would be to introduce a call like >> "getServerCapabilities()" which returns a bitmap, and define a bit per >> time that we add a new feature. >> >> The advantage of these approaches vs a single increasing version >> number is that we sometimes want to backport a new IPC to an older >> version, but not backport all of the intervening IPCs. Having a bitmap >> allows us to "pick and choose" on backports without having to pull in >> a bunch of things we didn't necessarily want. >> Good point. >> On Wed, Aug 1, 2012 at 1:41 AM, Stack <[email protected]> wrote: >>> On Tue, Jul 31, 2012 at 1:47 AM, Devaraj Das <[email protected]> wrote: >>>> Wondering whether we should retain the VersionedProtocol now that we have >>>> protobuf implementation for most (all?) of the protocols. I think we still >>>> need the version checks and do them when we need to. Take this case: >>>> 1. Protocol Foo has as one of the methods FooMethod(FooMethodRequest). >>>> 2. Protocol Foo evolves over time, and the FooMethod(FooMethodRequest) now >>>> has a better implementation called FooMethod_improved(FooMethodRequest). >>>> 3. HBase installations have happened with both the protocol >>>> implementations. >>>> 4. Clients should be able to talk to both old and new servers (and invoke >>>> the newer implementation of FooMethod if the protocol implements it). >>>> >>>> (4) is possible when the getProtocolVersion is implemented by the protocol >>>> at the server. The client could check what the version of the protocol was >>>> (assuming VersionedProtocol semantics where the protocol version number is >>>> upgraded for such significant changes) and depending on that invoke the >>>> appropriate method... >>>> >>>> Having to map version-numbers of protocols to the methods-supported is >>>> probably arcane IMO but works.. >>>> >>>> The other approach (that wouldn't require the version#) is to do something >>>> like - On the client side, get the protocol methods supported at the >>>> server (and cache it) and then look this map up whenever needed to decide >>>> which method to invoke. >>>> >>>> Any thoughts on whether we should invest time in the second approach yet? >>>> >>> >>> The VersionedProtocol w/ client being able to interrogate what methods >>> a server supports strikes me as a facility that will be rarely used if >>> at all and bringing it along, keeping up the directory of supported >>> methods, will take a load of work on our part that we'll do less than >>> perfectly so should it ever be needed, it won't work because we let it >>> go stale. >>> Yeah, this won't be a common case. It'd (hopefully) be rare. The directory of methods would be the methods in the protocol-interface at the server that could be figured by invoking reflection (and hence staleness issue shouldn't happen). >>> What do you reckon? >>> >>> The above painted scenario too is a little on the exotic side. We can >>> do something like Jimmy suggests in those rare cases we need to add a >>> new method because there is insufficient wiggle-room w/i the >>> particular PB method call (If we get into the issue Ted raises where >>> we'd have to go back to the server twice because there is a third new >>> method call, we're doing our API wrong). >>> Agree that the exception handling hack can be played here.. In general, having some solution around this might be really helpful *if* we get some API wrong (for e.g., indirect implication on memory by the API semantics) and we need to fix it without breaking compatibility.. In HDFS, listFile proved to be a memory killer for extremely large directories and people implemented the iterator version of the same. >>> The protocol needs a version though. We'll be still sending that >>> 'hrpc' long in the header preamble? Should we add a version long >>> after the 'hrpc' long? >>> The version in "hrpc" is the RPC version (as opposed to protocol version). I think that's orthogonal to this discussion.. >>> As to a directory of supported methods, do we need this in the >>> protocol at all? Can't this be knowledge kept outside of the >>> on-the-wire back and forth? >>> >>> St.Ack >> >> As I answered above, and as Todd also says, it probably makes sense to have a client wide cache for protocol<->supported-methods .. and look up the cache when and if the client needs to decide between different versions of a method, or picking a new method, based on the server it is talking to...
