On Aug 28, 2009, at 3:15 PM, Doug Cutting wrote:
Sanjay Radia wrote:
> No. The 1.0 proposal was that it included both API and wire
compatibility.
The proposal includes a lot of things, but it's so far just a
proposal.
There's been no vote to formally define what 1.0 will mean. In
every
discussion I've heard, from the very beginning of the project, it
primarily meant API stability. You've added wire compatibility, data
stability, security, restart recovery, etc. These are all very nice
features to have, essential perhaps in some contexts, but they may nor
may not be required for 1.0. I worry that if we keep piling more
things
on, we'll never get to 1.0.
Your email seems to suggest that I have added a whole bunch of ad-hoc
compatibility features over a period of time. Not true if you
check the record: API, Data Compatibility and Wire compatibility were
part of my proposal from the early days (see the email thread on this
subject). Data compatibility is already being supported and I was
assuming that we are going to continue to support it;
When I added Job Q compatibility, you commented that it was not
necessary at this stage and I replied on the Jira saying that I agreed
with you.
Surprised that you feel that wire compatibility was not even on the
table.
One of the key motivation for Yahoo developing Avro was wire
compatibility for hadoop 1.0.
It is okay to propose that we drop wire compatibility from 1.0, but to
state that this was never part of the discussion is misleading.
As far as voting; there was no vote - so that applies to all aspects
of the 1.0 proposal.
--------------
There are two related but somewhat separate issues here:
- what should and should not be part of 1.0
- how do we maintain the stability of the new interfaces that have
been marked as stable. (The reason
this email thread was initiated.) I think Doug, you are
suggesting that we addressed this one by calling 22 to be 1.0; a
reasonable proposal that we should discuss.
- I am not opposed to discussing something like this but I want
to better understand your concerns about wire compatibility.
Is it lack of resources in getting wire compatibility or that you
think it is low priority for the community?
Both Facebook (Dhruba tells me) and Yahoo are suffering badly from the
lack of wire compatibility - a major motivaiton
for Yahoo to develop Avro.
To summarize wrt to 1.0:
API compatibility - I suspect all will say yes.
Data compatibility - something that I believe we should continue to
maintain.
Job Q compatibility - doug and sanjay have said: not now, good idea
for later.
Wire compatibility - open question; but my thoughts are:
With the progress we have made on Avro so far I think there is a
very good chance to get wire compatibility in 22 which we
can then call 0.99 or 1.0. I think it is worth a shot.
Doug
