I have been looking at an alternative approach (i think its a little less optimal), but in thoughts it might be more acceptable for those not wishing this into the core jms client. It means the custom object serialisation is a optional add-on rather than coded into the core jms client, similar idea to the way spring wraps connection factories for caching connection, and others.
https://github.com/michaelandrepearce/activemq-artemis/tree/SerdesExtension/artemis-jms-client-extensions <https://github.com/michaelandrepearce/activemq-artemis/tree/SerdesExtension/artemis-jms-client-extensions> It essentially is wrapping at the JMS api level, so actually would/should work for any JMS Client not just artemis, e.g. could wrap QPID client. Thoughts on this approach rather than native integration to the clients? > On 1 Jun 2017, at 21:54, Matt Pavlovich <[email protected]> wrote: > > I understand the usefulness of adding this to the Client Connection Factory > vs kicking all the way up to an integration stack (like Camel). In my eyes, > its about solving for cross-platform object serialization using byte payloads > instead of the problems associated with non-portable/cross platform > ObjectMessage and MapMessage JMS message types. Tibco-RV was really good for > that as well. > > IMHO— plugging in protobuf, Avro, or other binary-serialization > flavor-of-the-month at the connection factory level has real value. > >> On Jun 1, 2017, at 2:23 PM, Clebert Suconic <[email protected]> >> wrote: >> >> I honestly don't see an issue on making the write and readObject a >> pluggable operation. It's a simple change on the API at >> ActiveMQConnectionFactory. >> >> On Thu, Jun 1, 2017 at 1:09 PM, Michael André Pearce >> <[email protected]> wrote: >>> This is why the proposal is for a pluggable interface to declared to >>> convert from Object to byte[] and back not saying one moment Artemis owns >>> or has all the implementations. >>> >>> Anyhow your points are taken on board, we def need to agree on a well >>> defined and clean api, to avoid exactly that situation. >>> >>> Sent from my iPhone >>> >>>> On 1 Jun 2017, at 17:59, Timothy Bish <[email protected]> wrote: >>>> >>>>> On 06/01/2017 12:49 PM, Michael André Pearce wrote: >>>>> Not at all that's the point >>>>> >>>>> Application code uses JMS Api. The serdes are just defined/declared into >>>>> the connection factory typically the only part of the app exposed to any >>>>> particulars about the broker implementation detail. >>>>> >>>>> Yes we can do converter.convert(object) in code, this is just manual code >>>>> in the app space. >>>>> >>>>> but like with kafka and I have to stress it's successfulness is that that >>>>> converter is in it's api of the product. Which means a lot of companies >>>>> reuse a few single implementations and a good eco system occurs like with >>>>> schema registry (Eg we use confluents serdes we don't code out own) >>>> >>>> Which is exactly what camel can solve and without starting down the >>>> slippery slop of packing the same data format conversions Camel can >>>> already handle into the Artemis code base as will happen as each user >>>> wants their own preferred data format. >>>> >>>> Similar things were tried in the ActiveMQ 5.x code and abandoned over time >>>> so I'd like to avoid that if possible. >>>> >>>> Anyway, I've raised my objection. >>>> >>>>> >>>>> >>>>> >>>>> Sent from my iPhone >>>>> >>>>>>> On 1 Jun 2017, at 17:39, Timothy Bish <[email protected]> wrote: >>>>>>> >>>>>>> On 06/01/2017 12:19 PM, Michael André Pearce wrote: >>>>>>> Agreed it does as an EAI pattern or flow, But I have to code/define >>>>>>> into Camel's dsl, it does JMS as much as our custom app code would it >>>>>>> consumes the JMS api. >>>>>> So you still need to code / define the serialization then and include >>>>>> that in you application which means the difference between some >>>>>> connection.setSerializationThing() vs >>>>>> producer.send(Converter.convert(payload)); so I don't see how that is a >>>>>> real value add to the JMS client. >>>>>> >>>>>>> What we propose here is just providing a clean way to define the JMS >>>>>>> ObjectMessage internal serialisation. If Java serialisation isn't your >>>>>>> cup of tea. (Which for many reasons isn't for us, and I'm sure it's >>>>>>> similar for others) >>>>>>> >>>>>> >>>>>> >>>>>>>>> On 1 Jun 2017, at 16:58, Timothy Bish <[email protected]> wrote: >>>>>>>>> >>>>>>>>> On 06/01/2017 11:50 AM, Michael André Pearce wrote: >>>>>>>>> Really this is much more about how an ObjectMessage serializes the >>>>>>>>> Object. As we have C++ clients etc that obviously won't be able to >>>>>>>>> understand Java serialized object. >>>>>>>>> >>>>>>>>> We use Avro and a schema repo for our dto transfer over the wire, >>>>>>>>> it's been a real performance boost , and removed some core data >>>>>>>>> issues, and really like to use it over the JMS land. >>>>>>>>> >>>>>>>>> One can argue that you could manually code this that you serialize >>>>>>>>> the data manually first and then just manually send a BytesMessage. >>>>>>>>> >>>>>>>>> But I think taking some inspiration from other places where a serdes >>>>>>>>> pattern is done has really helped (Kafka), from a corporation user >>>>>>>>> approach wiring some prebuilt serdes into a factory is very easy, >>>>>>>>> having duplicated code in many many apps leaves for issues, and >>>>>>>>> implementation divergence. >>>>>>>>> >>>>>>>>> The one downside of Kafka is it's lack of spec api, this is one big >>>>>>>>> sell of artemis as it's JMS compliant. Coding against JMS api for >>>>>>>>> Java estate is a huge win, this is suggesting taking some of the good >>>>>>>>> bits :). >>>>>>>>> >>>>>>>>> Does camel expose this as some sort of JMS API wrapper? I thought it >>>>>>>>> was much more an EAI solution. >>>>>>>>> >>>>>>>>> Cheers >>>>>>>>> Mike >>>>>>>> Camel does JMS transport: http://camel.apache.org/sjms.html >>>>>>>> Camel does AVRO: http://camel.apache.org/avro.html >>>>>>>> >>>>>>>> >>>>>>>>> Sent from my iPhone >>>>>>>>> >>>>>>>>>>>> On 1 Jun 2017, at 15:18, Martyn Taylor <[email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Jun 1, 2017 at 2:45 PM, Timothy Bish <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> On 06/01/2017 09:34 AM, Martyn Taylor wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Jun 1, 2017 at 2:32 PM, Timothy Bish >>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 06/01/2017 08:51 AM, Martyn Taylor wrote: >>>>>>>>>>>>>> I get the use case for using JSON/XML, particularly for cross >>>>>>>>>>>>>> language >>>>>>>>>>>>>> communication. >>>>>>>>>>>>>> >>>>>>>>>>>>>> One way users get around this problem right now is just to >>>>>>>>>>>>>> serialize >>>>>>>>>>>>>> to/from XML/JSON at the client application level and just use JMS >>>>>>>>>>>>>> TextMessages to send the data. I guess the idea here to remove >>>>>>>>>>>>>> that >>>>>>>>>>>>>> complexity from the client application and into the client via >>>>>>>>>>>>>> these >>>>>>>>>>>>>> pluggable serializer objects? Removing the serizliation logic >>>>>>>>>>>>>> out of >>>>>>>>>>>>>> code >>>>>>>>>>>>>> and into configuration. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Providing I've understood this properly, it seems like a good >>>>>>>>>>>>>> idea to >>>>>>>>>>>>>> me. >>>>>>>>>>>>>> so +1. >>>>>>>>>>>>>> >>>>>>>>>>>>>> This problem has already been solved via frameworks like Apache >>>>>>>>>>>>>> Camel, >>>>>>>>>>>>> putting such complexity into the JMS client is solving a problem >>>>>>>>>>>>> that's >>>>>>>>>>>>> already been solved and in much more flexible and configurable >>>>>>>>>>>>> ways. >>>>>>>>>>>>> >>>>>>>>>>>> Thanks Tim. I am not a Camel expert in any shape or form, how much >>>>>>>>>>>> additional complexity/configuration would be required to do >>>>>>>>>>>> something >>>>>>>>>>>> similar with Camel? My understanding of the proposal here is >>>>>>>>>>>> really just >>>>>>>>>>>> to give control back to the user in terms of how their objects are >>>>>>>>>>>> serialized. I'd expect this to be pretty light weight, just allow >>>>>>>>>>>> a user >>>>>>>>>>>> to configure a class to do the serialization. >>>>>>>>>>>> >>>>>>>>>>> Camel offers conversions for a number of data formats >>>>>>>>>> Sure. Though, one of the drivers (mentioned in this thread) for >>>>>>>>>> having >>>>>>>>>> control over the de/serialization process was for performance. >>>>>>>>>> Converting >>>>>>>>>> to another format is going to obviously make this much worse. >>>>>>>>>> >>>>>>>>>>> as well as routing amongst numerous protocols, have a look at the >>>>>>>>>>> supported data formats page: >>>>>>>>>>> http://camel.apache.org/data-format.html and >>>>>>>>>>> the transports http://camel.apache.org/transport.html >>>>>>>>>>> This doesn't seem to be doing much more for the user than moving >>>>>>>>>>> the work >>>>>>>>>>> they need to do around, >>>>>>>>>> Well, it abstracts the de/serialization process out of application >>>>>>>>>> code. >>>>>>>>>> >>>>>>>>>>> they still have to implement or configure the mechanics of the >>>>>>>>>>> transformation of the data format to the appropriate JMS message >>>>>>>>>>> type and >>>>>>>>>>> back again. Even if you bake in something to the client to handle >>>>>>>>>>> some >>>>>>>>>>> common formats you will quickly find that it doesn't meet >>>>>>>>>>> everyone's needs >>>>>>>>>>> and you'll end up implementing a poor mans Camel inside a JMS API >>>>>>>>>>> restricted client which seems less than ideal. >>>>>>>>>> I agree reinventing the wheel (badly) is not a good idea. So, if >>>>>>>>>> Camel is >>>>>>>>>> able to provide us with a solution to the problem, that addresses the >>>>>>>>>> issues outlined here. Then, we should certainly look into it. >>>>>>>>>> >>>>>>>>>> Cheers. >>>>>>>>>> >>>>>>>>>>>> On Thu, Jun 1, 2017 at 7:44 AM, Michael André Pearce < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think i might be getting the problem, use case you want to go >>>>>>>>>>>>>> for, >>>>>>>>>>>>>> which >>>>>>>>>>>>>> >>>>>>>>>>>>>>> is to possible serialise to JSON or XML, because they're >>>>>>>>>>>>>>> supported well >>>>>>>>>>>>>>> in >>>>>>>>>>>>>>> other languages like c++, which won't read a java serialised >>>>>>>>>>>>>>> object, >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>> say for XML you generate objects via an XSD which by default >>>>>>>>>>>>>>> aren't >>>>>>>>>>>>>>> serialisable, so you cannot simply add Serializable to the >>>>>>>>>>>>>>> object, as >>>>>>>>>>>>>>> its >>>>>>>>>>>>>>> generated at build. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is this the problem we need to solve? If so: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> To get around this normally the tools that generate objects for >>>>>>>>>>>>>>> serialisation from schema such as XSD do support a way to >>>>>>>>>>>>>>> toggle or >>>>>>>>>>>>>>> change >>>>>>>>>>>>>>> the generation slightly for some common use cases. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In case of XSD, where using jaxb it would be to add something >>>>>>>>>>>>>>> like the >>>>>>>>>>>>>>> below to jaxb global bindings: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> <xs:annotation> >>>>>>>>>>>>>>> <xs:appinfo> >>>>>>>>>>>>>>> <jaxb:globalBindings generateIsSetMethod="true"> >>>>>>>>>>>>>>> <xjc:serializable uid="12343"/> >>>>>>>>>>>>>>> </jaxb:globalBindings> >>>>>>>>>>>>>>> </xs:appinfo> >>>>>>>>>>>>>>> </xs:annotation> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> like wise if you are generating POJO's from a jsonschema using >>>>>>>>>>>>>>> for say >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> tool jsonschema2pojo there is a toggle in the maven plugin >>>>>>>>>>>>>>> serializable >>>>>>>>>>>>>>> which you can switch to true. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Obviously if you hand crank your DTO Pojo's then it's a case of >>>>>>>>>>>>>>> simply >>>>>>>>>>>>>>> add >>>>>>>>>>>>>>> implement Serializable to the class. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cheers >>>>>>>>>>>>>>> Mike >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Sent from my iPhone >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 1 Jun 2017, at 06:57, Michael André Pearce < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> we could but then it wouldn't work via jms api. Typically if >>>>>>>>>>>>>>> using jms >>>>>>>>>>>>>>>> the only custom or specific broker object is the connection >>>>>>>>>>>>>>>> factory >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> rest you code to Jms. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Sent from my iPhone >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 1 Jun 2017, at 04:10, Clebert Suconic >>>>>>>>>>>>>>>> <[email protected]> >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> On Wed, May 31, 2017 at 10:47 PM Michael André Pearce < >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Jms api dictates class set in object message to be >>>>>>>>>>>>>>>>> serializable. >>>>>>>>>>>>>>>>> We could make an extension. It could be an extra message this >>>>>>>>>>>>>>>>> actually. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 31 May 2017, at 22:37, Timothy Nodine >>>>>>>>>>>>>>>>> <[email protected]> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Should the interface require the underlying class to be >>>>>>>>>>>>>>>>> Serializable? >>>>>>>>>>>>>>>>> One use case might be to provide serialization to classes >>>>>>>>>>>>>>>>> that aren't >>>>>>>>>>>>>>>>>> natively serializable. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Michael André Pearce wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> To help discussion, >>>>>>>>>>>>>>>>>>>> A very very basic implementation just to simulate the idea. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> https://github.com/michaelandrepearce/activemq-artemis/tree/ >>>>>>>>>>>>>>>>>> CustomSerialisation >>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> https://github.com/michaelandrepearce/activemq-artemis/tree/ >>>>>>>>>>>>>>>>>> CustomSerialisation >>>>>>>>>>>>>>>> n.b. doesn’t fully compile is just pseudo impl, nor doesn’t >>>>>>>>>>>>>>>> include >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> bits as discussed below like map/change type to a byte >>>>>>>>>>>>>>>>> message for >>>>>>>>>>>>>>>>>> compatibility, nor media type idea. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Cheers >>>>>>>>>>>>>>>>>>>> Mike >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Clebert Suconic >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>> Tim Bish >>>>>>>>>>>>> twitter: @tabish121 >>>>>>>>>>>>> blog: http://timbish.blogspot.com/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Tim Bish >>>>>>>>>>> twitter: @tabish121 >>>>>>>>>>> blog: http://timbish.blogspot.com/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>> -- >>>>>>>> Tim Bish >>>>>>>> twitter: @tabish121 >>>>>>>> blog: http://timbish.blogspot.com/ >>>>>>>> >>>>>> -- >>>>>> Tim Bish >>>>>> twitter: @tabish121 >>>>>> blog: http://timbish.blogspot.com/ >>>>>> >>>> >>>> -- >>>> Tim Bish >>>> twitter: @tabish121 >>>> blog: http://timbish.blogspot.com/ >>>> >> >> >> >> -- >> Clebert Suconic >
