Doug - thanks again. I agree with you. I have been looking into it for the past few days and it seems like this will require quite a bit of refactoring. I will try to follow up on Jira with more specific details.
Thanks Pankaj On Wed, Feb 27, 2013 at 2:50 PM, Doug Cutting <[email protected]> wrote: > Pankaj, > > Avro RPC is currently specified to always uses the binary encoding > (like Avro data files). This might reasonably be extended to JSON. > First we'd need to specify the new wire format. Probably Avro's > framing would not make sense for JSON-encoded RPC over HTTP. Then > we'd need to figure what of the existing Java implementation might be > reused or adapted. At a glance, it doesn't look to me like a few > one-line changes would suffice, adding methods where things are > hardwired, that rather more substantial changes would be required, but > I might be missing something. If you're interested in pursuing this > then please file an issue in Jira where you can propose changes to the > specification and implementation. > > Cheers, > > Doug > > On Wed, Feb 27, 2013 at 11:16 AM, Pankaj Shroff <[email protected]> wrote: > > I guess my question is more basic - given that this is somewhat specific > to > > my own use case: > > > > How does one use other forms of Encoder/Decoder implementations that are > > available in the Avro library along with the Avro-Ipc SDK. > > > > As of 1.7.3, I see that the only Encoding/Decoding that Avro-ipc > supports is > > the BinaryEncoding > > > > Pankaj > > > > > > > > On Mon, Feb 25, 2013 at 2:25 PM, Pankaj Shroff <[email protected]> > wrote: > >> > >> Doug > >> > >> Perhaps you answered a portion of my conundrum in another thread > >> (permalink below) - but there is still the handshake and reuse of > invocation > >> logic question. Let me also think about this a little bit. > >> > >> Thanks in any case. Avro is a great tool in any case! > >> > >> > >> > http://mail-archives.apache.org/mod_mbox/avro-user/201302.mbox/%3CCALEq1Z_rt8FasjSR%2B%2BOOgE3ogrAh0Y%2BtL3z47hznuiBAtfvWmw%40mail.gmail.com%3E > >> > >> > >> Pankaj > >> > >> [email protected] > >> > >> > >> > >> On Mon, Feb 25, 2013 at 1:38 PM, Doug Cutting <[email protected]> > wrote: > >>> > >>> This sounds like a different RPC wire format than Avro's. Avro's > >>> Requestor and Responder implement Avro's RPC wire format. Avro's > >>> Encode/Decoder and DatumReader/DatumWriter APIs should facilitate > >>> implementation of other RPC wire formats that include Avro data. > >>> Avro's Transceiver API may or may not be reusable, since it assumes > >>> Avro-style framing. Parts of Requestor and Responder *might* be > >>> reusable and some refactoring of those classes *might* make such reuse > >>> easier, but there's not that much logic there that's not specific to > >>> Avro's wire format, so it might be just as easy to reimplement this > >>> layer for a different wire format. It's hard for me to say without > >>> seeing a patch with a proposed refactoring. Does that make sense? > >>> > >>> Doug > >>> > >>> On Mon, Feb 25, 2013 at 9:08 AM, Pankaj Shroff <[email protected]> > wrote: > >>> > Hi > >>> > > >>> > We are using Avro for implementing an open source reference > >>> > implementation > >>> > of the OpenRTB protocol. > >>> > > >>> > We have made a design goal to model the protocol using Avro protocol > >>> > files > >>> > (.avpr) and generate types defined in the protocol schema using Avro > . > >>> > The > >>> > challenge is that the protocol does not necessarily require the use > of > >>> > Avro/ > >>> > Binary wire encoding - or even the use of Avro/ RPC context. In fact > >>> > many > >>> > counter parties have proprietary implementations supporting either > >>> > Protobuf > >>> > or Json encoding. > >>> > > >>> > Now, there is a Json encoder/decoder in the Avro package but it seems > >>> > that > >>> > the approach is a "schema-first" approach. The JsonEncoder assumes > that > >>> > the > >>> > encoding on the wire still follows the Avro Json encoding - which > >>> > includes a > >>> > handshake followed by schema confirmation on both sides (client and > >>> > server). > >>> > > >>> > For the protocol we are implementing - this presents 2 problems if > >>> > Avro/ > >>> > binary is not the chose encoding type for both sides - and if > instead, > >>> > lets > >>> > say, raw Json encoding is being used > >>> > > >>> > 1) the handshake is rather Avro specific - and we would like to > >>> > completely > >>> > skip it if both sides have agreed on using raw json encoding - there > >>> > may be > >>> > a separate info-exchange phase in the protocol that is protocol > >>> > specific and > >>> > does not need to use Avro handshake. Is it possible to use Avro RPC > >>> > without > >>> > the handshake? > >>> > > >>> > 2) we would like to use the data binding and schema resolution as > >>> > implemented by the SpecificResponder class in Avro - but extend it to > >>> > use > >>> > raw JSON - not Avro JSON - encodings. > >>> > > >>> > 3) We would prefer not to have to override the > "respond(List<buffers>)" > >>> > method of the base class Responder. This implementation always > performs > >>> > handshake and always uses BinaryEncoder/Decoder which removes any > >>> > flexibility of using a different encoder /decoder in a derived class. > >>> > We > >>> > would prefer if the Responder or some derived class saves the chosen > >>> > Decoder/ encoder as a protected property of the Responder object. > >>> > Instead of > >>> > instantiating BinaryEncoder/ Decoder objects on the fly within the > >>> > respond > >>> > method, it would be great if this was made more extensible and if the > >>> > Encoder/Decoder can be specified during construction. > >>> > > >>> > 4) For future flexibility it would be great to have the handshake > >>> > functionality available in sub-classes of Responder as an inherited > >>> > method > >>> > (instead of private scope right now). > >>> > > >>> > I would welcome any suggestions/corrections. > >>> > > >>> > Pankaj > >>> > > >>> > -- > >>> > Pankaj Shroff > >>> > twitter: @chompi > >>> > http://github.com/chompi > >>> > http://github.com/openrtb/openrtb2x > >>> > > >> > >> > >> > >> > >> -- > >> Pankaj Shroff > >> [email protected] > > > > > > > > > > -- > > Pankaj Shroff > > [email protected] > -- Pankaj Shroff [email protected]
