I'm -1 on avro as a RPC format. Thrift is the way to go, any of the advantages of smaller serialization of avro is lost by the sheer complexity of avro and therefore the potential bugs.
I understand the desire to have a pluggable RPC engine, but it feels like the better approach would be to adopt a unified RPC and just be done with it. I had a look at the HsHa mechanism in thrift and it is very good, it in fact matches our 'handler' approach - async recieving/sending of data, but single threaded for processing a message. -ryan On Fri, May 27, 2011 at 1:00 PM, Andrew Purtell <[email protected]> wrote: > Also needing, perhaps later, consideration: > > - HDFS-347 or not > > - Lucene embedding for hbase-search, though as a coprocessor this is already > pretty much handled if we have platform support (therefore a platform module) > for a HDFS that can do local read shortcutting and block placement requests > > - HFile v1 versus v2 > > Making decoupled development at several downstream sites manageable, with a > home upstream for all the work, while simultaneously providing clean > migration paths for users, basically. > > --- On Fri, 5/27/11, Andrew Purtell <[email protected]> wrote: > >> From: Andrew Purtell <[email protected]> >> Subject: modular build and pluggable rpc >> To: [email protected] >> Date: Friday, May 27, 2011, 12:49 PM >> From IRC: >> >> apurtell i propose we take the build modular as early as possible to deal >> with multiple platform targets >> apurtell secure vs nonsecure >> apurtell 0.20 vs 0.22 vs trunk >> apurtell i understand the maintenence issues with multiple rpc engines, >> for example, but a lot of reflection twistiness is going to be worse >> apurtell i propose we take up esammer on his offer >> apurtell so branch 0.92 asap, get trunk modular and working against >> multiple platform targets >> apurtell especially if we're going to see rpc changes coming from >> downstream projects... >> apurtell also what about supporting secure and nonsecure clients with the >> same deployment? >> apurtell zookeeper does this >> apurtell so that is selectable rpc engine per connection, with a >> negotiation >> apurtell we don't have or want to be crazy about it but a rolling upgrade >> should be possible if for example we are taking in a new rpc from fb (?) or >> cloudera (avro based?) >> apurtell also looks like hlog modules for 0.20 vs 0.22 and successors >> apurtell i think over time we can roadmap the rpc engines, if we have >> multiple, by deprecation >> apurtell now that we're on the edge of supporting both 0.20 and 0.22, and >> secure vs nonsecure, let's get it as manageable as possible right away >> >> St^Ack_ apurtell: +1 >> >> apurtell also i think there is some interest in async rpc engine >> >> St^Ack_ we should stick this up on dev i'd say >> >> Best regards, >> >> - Andy >> >> Problems worthy of attack prove their worth by hitting >> back. - Piet Hein (via Tom White) >> >
