I like the idea of pluggable serialization/marshalling, but as you rightly explained, what flexibility gives you you lose by lack of functionality. E.g. querying is only available for protostream based encoding.
So, I think we need to remove the hard bind between functionality and encoding to be able to have a trully pluggable encoding mechanism. Cheers, -- Galder Zamarreño Infinispan, Red Hat > On 3 Feb 2016, at 11:38, Gustavo Fernandes <[email protected]> wrote: > > > > On Mon, Jan 25, 2016 at 2:03 PM, Galder Zamarreño <[email protected]> wrote: > Hi all, > > As I write the Javascript client for Hot Rod, and Vittorio writes the C++ > client, the question the encoding of the byte arrays has popped up. > > The reason why encoding matters is mainly because of compatibility mode. How > does a Hot Rod client know how it should transform something a REST client > set? > > To be able to answer this question correctly, the Hot Rod client needs to > know the type of the data. > > Although we could consider adding encoding information to the Hot Rod > protocol long term, in the short term this question might already been > answered by Protostream. > > > Compatibility between disparate clients should certainly be supported, but > what about a pluggable marshaller mechanism? Let's consider all usage > scenarios: > > * Only Java clients: in most cases, JBoss marshalling is adequate, no need to > involve protobuf or json and since JBoss Marshalling is default, no need to > configure anything. > > * Mix of Java and C++/C#/Python clients: the Protobuf encoding works > wonderfully for that. The client could be configured to use the protostream > marshaller, the same for the server. > > * Mix of Java, C++/C#/Python and REST clients: protobuf with json [1] > encoding is an interesting option. Same as above, a protostream marshaller > with json support could > be configured both on client and server. > > * Custom marshallers: consider FlatBuffers [2], for example, an interesting > new cross-platform serializer that allows to access serialized data without > de-serializing the whole payload, > and without generating any transient memory, and it optionally supports JSON. > It could be interesting to write a marshaller based on it and plug it on > Infinispan. > > Although we have a way of configuring the marshaller on client (via > RemoteCacheManager) and server (deployable jar), there'd be more work to do > in order make them really pluggable: > > - Some serialization formats like Protobuf and [2] require interface > description to be available on both client and server. This is not supported > for all marshallers, just for Protostream > - Remote query requires some metadata regarding which fields to index and > how, and this would need to be possible on every language regardless of the > marshalling. > Currently, this info can be encoded in the .proto file only, a custom > marshaller would not work. > - Currently remote query is settled on protobuf not only for the byte array > but for the the HotRod query request and response [3] > > Is having a pluggable serializer mechanism too far fetched, given the effort > already invested towards protobuf? > > Gustavo > > [1] https://developers.google.com/protocol-buffers/docs/proto3#json > [2] https://google.github.io/flatbuffers/ > [2] > https://code.facebook.com/posts/872547912839369/improving-facebook-s-performance-on-android-with-flatbuffers/ > [3] > https://github.com/infinispan/infinispan/blob/master/remote-query/remote-query-client/src/main/resources/org/infinispan/query/remote/client/query.proto > > > > > _______________________________________________ > infinispan-dev mailing list > [email protected] > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > [email protected] > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list [email protected] https://lists.jboss.org/mailman/listinfo/infinispan-dev
