Kevin Clark wrote:
The overhead for those people (or some equivalent group) to pay attention to another mailing list, another bug tracker, another irc channel, and another community isn't trivial.
Communities form around code, and, if Avro's code is largely disjoint from Thrift's, we should not assume that everyone in the Thrift community cares about Avro or vice versa.
Of course, this assumes that one of the primary goals of Avro is to be cross language. Is that the case, or have I misunderstood?
Yes, that is a goal.
It would be perfectly reasonable for Hadoop to specify that they use the Avro data format for transmissions, and the cross language library to provide the API could be Thrift. I think you said something similar in your post, but if not please do clarify.
Yes, perhaps this could be done. I am not convinced that TProtocol is an ideal API for reading and writing Avro data, but it could perhaps be made to work reasonably well.
That being said, I'm fairly confident we'll be providing an Avro protocol on our own at some point if you're not interested in working together. But I think if we go down that path we're doing a disservice to users of both Thrift and Avro.
I have never said I was not interested in working together. I've said that I think Avro is fundamentally different from Thrift. Avro is a specific format, Thrift is a generic API for various formats, none like Avro. They might be made to work together. But at this point I see no point in forcing them together. If TProtocol's API is a good match for Avro's format and features, then it should be easy for folks to implement TProtocol using Avro's code and include Avro in Thrift. If the match is not good then perhaps we can adjust Thrift and/or Avro to improve it.
Doug
