If my understand is correct, this decision is an easy one to make, because whatever we choose, we still comply with the specification.
Avatica specifies its API at two levels: (1) the Java API (Meta), (2) the JSON requests/responses going over the wire. I would accept any transport that complies with both of those levels. In principle we could write a JSON-over-HTTP-marshalled-via-Jackson transport, specifying the mappings elsewhere than in the Meta classes, and someone else could use another transport. Is it feasible to achieve this? Julian > On Aug 7, 2015, at 5:40 PM, Josh Elser <[email protected]> wrote: > > Hrmph. Well, fun fact: that JSON serialization for pb3 apparently isn't > implemented yet.. > > Josh Elser wrote: >> :) I hear you. I don't think I have anything putting me in one camp or >> the other aside from potential worries. >> >> I can respect the Thrift "doing too much" notion -- kind of half of my >> concern in using something like Thrift just to get object serialization >> stuff (now I'm thinking I should help separate that out within thrift!) >> I've used Thrift since the 0.6 timeframe and I don't remember once >> coming across an issue where object serialization wasn't handled correctly. >> >> I'll have to poke around pb3 docs some more to see if I can find some >> talk on compatibility. I think at this point I'm leaning towards >> protobuf, but we'll see. >> >> Thanks again for taking the time to write up your thoughts, Jesse! >> >> Jesse Yates wrote: >>> Yup, totally get it. Honestly, I have no idea what the compat story is >>> supposed to be for proto3. At the same time, I've got no hands on >>> experience w/ Thrift either, beyond dabbling a little bit years ago, so I >>> don't know what their on-the-wire-compat story is either :) I'm not as >>> huge >>> of a fan of thrift in general though because it tries to do too much IMO >>> >>> If we do go the thrift route, but want to support grpc, we could do the >>> work to make a thrift adapter, so picking one or the other now doesn't >>> really preclude doing cool stuff later. >>> >>> In short, it doesn't really matter? At least not to me. >>> >>> On Thu, Aug 6, 2015 at 3:57 PM Josh Elser<[email protected]> wrote: >>> >>>> Thanks for the feedback, Jesse! >>>> >>>> To clarify, the compatibility I'm most worried about is backwards compat >>>> in future versions of protobuf3. I hadn't considered v2 support into the >>>> equation -- figured v3 is the starting point like you say. I know the >>>> protobuf 2.4 to 2.5 upgrade in Hadoop/HBase land was rather scary that >>>> required a lot of lock-step movement across many big projects. This is >>>> my biggest fear with protobuf. >>>> >>>> Despite fear of derailing conversation, grpc definitely has some nice >>>> features built in (and implementing a service is definitely slick). I >>>> think it's definitely a "later" since the single endpoint presently used >>>> doesn't really require a complex service definition (with evolving args, >>>> etc). My gut reaction is to wait to adopt such software until there is a >>>> definite understanding of what the service would look like (which I >>>> assume will become clearer as development continues). >>>> >>>> Jesse Yates wrote: >>>>> +1 for proto3. mostly for the later move to using grpc, which lets us >>>>> leverage a battle tested framework for doing real protobuf services >>>>> with >>>>> fast streaming (how nice!). >>>>> >>>>> proto3 can be backwards compat in some ways (eg. map implementation >>>>> done >>>>> manually in proto2). I don't think its a huge issue since there are >>>>> currently no backwards compat guarantees anyways AFAIK. >>>>> >>>>> >>>>> On Thu, Aug 6, 2015 at 3:11 PM Josh Elser<[email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I have some ideas which I'd like to try to working on to improve >>>>>> Avatica. My direct interest is in support of the Phoenix Query Server >>>>>> which builds on top of Avatica. >>>>>> >>>>>> One of the big areas of improvement that stands out to me is the >>>>>> Jackson-annotated Response/Request classes. First, we'd want to make >>>>>> sure these classes are stable (in terms of fields/attributes) so that, >>>>>> as each object evolves, we prevent existing clients from breaking. >>>>>> Second, it's desirable that we have some representation of these >>>>>> objects >>>>>> in a manner that's accessible to non-Java clients. >>>>>> >>>>>> Ideally, preserving JSON as the serialized representation of these >>>>>> classes is also desirable as it's always pleasant to actually be >>>>>> able to >>>>>> read the data going over the wire when necessary. >>>>>> >>>>>> Given these goals, two tools stood out to me as candidates that would >>>>>> solve this problem well without extra effort in the transport: Thrift >>>>>> and Protobuf v3. >>>>>> >>>>>> Using Thrift only for object generation is a bit overkill, but this is >>>>>> an area of Thrift that has been rock-solid in the years that I've used >>>>>> it. Classpath issues with Thrift are always a concern as multiple >>>>>> versions of Thrift tend to break terribly (at least when performing >>>>>> Thrift RPCs -- I wouldn't necessarily trust it to be safe for only >>>>>> object serialization). It's easy to serialize Thrift objects to >>>>>> JSON and >>>>>> we can generate these classes for other languages. >>>>>> >>>>>> I would call Protobuf v3 very young, although it does appear to fit >>>>>> the >>>>>> bill. Protobuf v2 has had a track record of being a pain when >>>>>> different >>>>>> versions of the library exist. For example, the Java classes generated >>>>>> by protobuf 2.x would fail to work with protobuf 2.y. I assume there's >>>>>> not enough evidence on protobuf v3 to say whether this will >>>>>> continue to >>>>>> be a problem. JSON-representation of objects and cross-language >>>>>> support >>>>>> are both natively supported (best as I can tell). Hypothetically, >>>>>> adopting protobufv3 could also have some extra benefits down the road >>>>>> such as leveraging grpc to replace Jetty, but that is an entirely >>>>>> different discussion for later :) >>>>>> >>>>>> If you've gotten this far, I'd humbly ask for any opinions on the >>>>>> technology choices. Any opinions for or against one or the other? Any >>>>>> similar experiences? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> - Josh >>>>>> >>>
