I'm writing a client server application, and have been reading about Avro,
looking at it's sources etc. Basically I want to be able to define a protocol
from multiple modules/plugins or use different protocols per each module
(probably makes more sense based on the way responders seem to work...wrapping
a
transceiver). I want different modules/plugins to handle different client
requests to segregate the business logic allowing me to add plugins to the
system and extend it over time updating pieces individually: what Eclipse and
NetBeans RCPs are good at.
I wound up studying Avro more closely as its documentation about dynamic
languages etc seems it would have a higher possibility of being able to support
such a thing compared to ProtoBuff or Thrift. Not sure on that yet but checking
it out.
I haven't found anything in Avro to really support this well on the server side
right out of the box though the client side seems to already be well positioned
to do it. The issue is in the classloading and resolution. The Avro module
won't
know about the possible classes. As an example think of a business application
which can manage customers, notes, phone calls, and quotes. A quote or a note
won't specifically know about phone support. Customer support won't know about
any of the others though some other module might know how to link customers to
calls and quotes. All the customer management interface will really know about
it managing customer information. These will all have separate classloaders.
On the server side is where this seems to get complicated with Avro. That may
of
course be where I'm just starting with it and learning it though.
I have had different thoughts. I'm wanting to use as much of Avro which is
already available as possible. The top two seem to be:
1) Write my own server implementation which handles multiple responders. Before
handing off to a responder the server would inspect the binary format and
decide
on which responder to use. Using the NettyServer sources I'm thinking this
would
inject some where around here:
List<ByteBuffer> req = dataPack.getDatas();
List<ByteBuffer> res = responder.respond(req);
inspecting the ByteBuffer's directly. The responders would be obtained from a
provider. The providers can be looked up by namespace etc and this is how I'm
envisioning handling module/protocol support segregation; perhaps by protocol
and hash instead...something like that. The lookup system is injected into by
any provider module which places itself there.
The major snag I'm having here seems related to how this would work out with
the
the way responders and the handShake work and ByteBuffer lists. i.e. How do I
route the messages to the correct backend/server side responder? I'm thinking
in
this case I can do some minor parsing of the finite list of ByteBuffer
instances
I have representing a given message. Is there already tooling for such a thing?
Seems if I figure this out I'll have to copy some existing code to get going
more quickly at first. Would be interested in providing patches and code etc to
the project if this route is required. Per the way the client APIs seem defined
as it relates to SpecificResponder this would make more sense for such an idea.
2) Write a responder implementation which takes multiple providers and builds a
unified protocol to be parsed. When modules are perhaps live updated in the
server send some message telling all clients that connections must be
reconnected unless the updates require client updates which can force them to
restart in which case they need to reconnect anyways. After this message has
been sent out, close all connections from the server side, rebuild the protocol
from available providers, restart the server. Clients will have a period of
time
before they timeout after trying once receiving the message.
The snag here seems to be on the client side. It isn't straight forward exactly
how the client protocols which were merged on the server would react to this.
Perhaps it isn't a big deal if some how versions are the same and the records
and messages match up. Not sure exactly as I'm just beginning.
But, those are my two main ideas; not saying they are the best, but trying to
give someone more insight into the overall goal etc. I'm going to start with
just a simple prototype to learn more of course, but thought I should go ahead
and get a feeler out before I get too deep, and too figured it might get me
going faster if someone already has something similar they are willing to share
some details about.
Too, where is the best place to search the Avro mailing list archives? There
seemed to be no links to search the lists from the site (at least none that
jumped out).
Thank you, and I appreciate any help,
Wade
==================
Wade Chandler
Software Engineer and Developer
NetBeans Dream Team Member and Contributor
http://wiki.netbeans.org/wiki/view/NetBeansDreamTeam
http://www.netbeans.org
