Makes sense to me. I think it may be useful to check in the .avpr files that are induced on the way, to let folks start trying to use different clients for certain operations.
-- Philip On Wed, Nov 11, 2009 at 11:21 AM, Doug Cutting <[email protected]> wrote: > Justin Santa Barbara wrote: > >> What about Philip's point on existing Hadoop interfaces? Any plans for >> how >> we'll generate the Protocol object for these? >> > > I'm hoping to use reflection initially for this. That's the motivation for > my renewed interest in AVRO-80. > > https://issues.apache.org/jira/browse/AVRO-80 > > My rationale is that I don't want to assume we'll move Hadoop onto Avro > overnight. So I'd like to move things in a way that's easy to maintain in a > branch/patch. If we can get reflection to work, then we only need to update > two places per Hadoop protocol: where it calls RPC.getProxy() and > RPC.getServer(). Then we can start looking at performance. We cannot > commit Avro-based Hadoop RPC until performance is adequate, and we don't > want to have to maintain a patch that changes many central data structures > in Hadoop while we're testing and improving performance, since that might > take time. > > Once we've committed Hadoop to using Avro, then we can consider, > protocol-by-protocol, replacing Hadoop's Writable objects with Avro > generated objects. Until then, the protocol will be defined implicitly by > Java through reflection. > > Note that if performance is inadequate due to reflection itself, rather > than the client/server implementations, we might resort to byte-code > modification to accelerate it. > > https://issues.apache.org/jira/browse/AVRO-143 > > This would also be a temporary approach. Longer-term we should move Hadoop > to use protocols declared in .avpr files and generated classes. But I don't > think that's practical in the short-term. > > My current short-term goal is to try to get Avro's reflection to the point > where it can implement NamenodeProtocol. > > Does this make sense? > > Doug >
