Hi Devaraj, HBASE-5394 is resolved. Now HBaseRPC can support both Writable and PB.
There will be a swat team meeting for this next week. Greg will send out the meeting time and place soon. You, and all developers/users interested in this are welcomed to join, discuss and come up an agreement. Thanks, Jimmy On Thu, Feb 16, 2012 at 12:48 PM, Todd Lipcon <[email protected]> wrote: > Hi Devaraj, > > IMO, the "protocol translator" interface introduced in HDFS has been > more pain than it's worth, given pluggability seems like a non-goal > for us. I think it's easier to just tie ourselves to protobufs. In > essence the client library (e.g. HTable) acts as the translator > between the user types and the wire types (protobuf). > > That said, we should be absolutely sure that we don't expose protobufs > to the *client API* > > -Todd > > On Thu, Feb 16, 2012 at 12:30 PM, Devaraj Das <[email protected]> > wrote: > > Good writeup, Jimmy (was away for a few days due to an event in my > family) > > > > Some quick questions - Has there been any thoughts on the plan to use > HBASE-5394. Are we going to make the hbase protocols (like > HRegionInterface) protobuf aware? > > > > In Hadoop, I have seen the following: > > 1. In HDFS, the protocol definitions are not changed (like > org.apache.hadoop.hdfs.protocol.ClientProtocol). Instead there are > translators that are defined that implement the mapping of protobuf > datastructures to application-level datastructures and vice versa (for > example, have a look at ClientNamenodeProtocolTranslatorPB and > ClientNamenodeProtocolServerSideTranslatorPB in the package > org.apache.hadoop.hdfs.protocolPB). > > 2. In Yarn (MRV2), all protocol definitions are written in PB > > > > Since the base RPC still uses writables for payload encoding, a > translation happens when the protobuf objects are sent/received (as an > example look at org.apache.hadoop.ipc.ProtobufRpcEngine; classes > RpcRequestWritable and RpcResponseWritable). > > > > What does the HBase community think about the above? > > > > > > On Feb 13, 2012, at 1:02 PM, Jimmy Xiang wrote: > > > >> I posted the proposal on wiki: > >> > >> http://wiki.apache.org/hadoop/Hbase/HBaseWireCompatibility > >> > >> Thanks, > >> Jimmy > >> > >> On Mon, Feb 13, 2012 at 11:03 AM, Ted Yu <[email protected]> wrote: > >> > >>> Can you post on wiki ? > >>> > >>> Attachment stripped. > >>> > >>> On Mon, Feb 13, 2012 at 11:01 AM, Jimmy Xiang <[email protected]> > wrote: > >>> > >>>> Hello, > >>>> > >>>> As HBase installation base is getting bigger, we are ready to work on > the > >>>> wire compatibility issue. > >>>> The goal is to make HBase easier for operators to upgrade, while it is > >>>> also easier for developers to > >>>> enhance, re-architect if necessary. > >>>> > >>>> The attached is a proposal we came up. We'd like to start with two > >>> phases: > >>>> > >>>> Phase 1: Compatibility between client applications and HBase clusters > >>>> Phase 2: HBase cluster rolling upgrade within same major version > >>>> > >>>> Could you please review? > >>>> > >>>> Thanks, > >>>> Jimmy > >>>> > >>> > > > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
