Re: STOMP and JMSType
On 6/13/06, Nathan Mittler [EMAIL PROTECTED] wrote: So it sounds like we're all in agreement on the content-type header. For text, it would be something like text There could be a few values of Content-type which map to text (text/xml, application/soap, application/xml etc). Incidentally the default implementation for sending an ObjectMessage / MapMessage to a Stomp client could be to use XStream to turn it into XML and mark it as text/xml. Otherwise its gonna be extremely hard for a typical Stomp client to read the message. and for bytes it would be application/octet-stream. So this would not be an application-level header, but would be used by my stomp client code to determine which message type to create. If we're all in agreement with that, then it seems to make sense that the default functionality of the broker be modified to handle content-type in this way. And if that's true, then it seems like this particular issue is resolved. This way, we get it into the 4.1 release with no problems. We can create another issue to do the refactoring as you've suggested ... which will probably take a little more time and several conversations to get right. How does this sound? Sounds great. -- James --- http://radio.weblogs.com/0112098/
Re: STOMP and JMSType
On 6/14/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 13, 2006, at 3:06 PM, Nathan Mittler wrote: So it sounds like we're all in agreement on the content-type header. For text, it would be something like text and for bytes it would be application/octet-stream. So this would not be an application-level header, but would be used by my stomp client code to determine which message type to create. Content-type is application level. I was suggesting it for your use case where you want to know what to convert a bytes message into in your C++ library =) Agreed! the app in this use case is the C++ client lib. so the c++ client lib could set the content-type to control a custom transformer so that sent JMS messages are exactly as the C++ stomp lib wants them to look like. So I'm starting to think there are 2 main use cases: 1) I want to have portable STOMP client that work on other providers. Then you accept that you can not tightly integrate with an existing JMS network in a portable way. For example they would not be able to send and receive JMS Map messages. Since stomp does not specify what those messages would look like on the wire. This means that STOMP needs to define how a portable mapping of JMS ByteMessage and TextMessage to STOMP Messages. I think what we have today is very close to this. We may just need to formalize it a little more in a document so that other providers could implement the same STOMP to JMS mapping. Of course, this mapping has to stay simple. 2) You have a STOMP client that does not mind intimately knowing about ActiveMQ. Then it can request transformation on the the send and receives. That transformation could totally change all the STOMP rules about the headers for for the messages coming in and out. It might use the content-type to hold the JMS message type: bytes, text, object, map, etc. and other headers like jms.Type to hold the JMSType headers. Also the payload encoding could be fancier. So by default, I think it should work like case #1, if you want to use case#2, then you use the transform header options. This gives us backward compatibility but for your C++ stomp client that exposes a JMS like API use case, I would think it falls under use case #2. Also, as offshoot idea on #2, is a provider allowed able to expose additional verbs/operations to clients? Kinda like how SVN/DAV does for HTTP? If we're all in agreement with that, then it seems to make sense that the default functionality of the broker be modified to handle content- type in this way. No. activemq-transformer would provide the JMS type mapping to override the default. I suggested that you use content-type (not required by stomp) to decide if something is text or a byte stream. Transfer-encoding is also useful for this. :-) And if that's true, then it seems like this particular issue is resolved. This way, we get it into the 4.1 release with no problems. We can create another issue to do the refactoring as you've suggested ... which will probably take a little more time and several conversations to get right. How does this sound? Nate On 6/13/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 13, 2006, at 1:50 PM, Nathan Mittler wrote: Could you guys point me to a place in AMQ where this sort of thing is being done? That would save me a lot of searching =) I'm viewing this problem from the client side - the Stomp C++ client that Tim Bish and I are writing currently supports text and bytes messages. Within a general Stomp client, I would suggest that switching on JMS message types is not a productive goal. Using Content-type here makes a lot more sense, I think. . It would make a lot of sense to set it for text messages going out to Stomp if there is not one already supplied. Stomp is a protocol, like HTTP or SMTP -- hardwiring a client to a specific server implementation is probably not a general solution (though is fine if it is specifically an activemq client which happens to use stomp for transport). So when I get a stomp frame in, I need to map it back to a text or bytes message. We chose to do this for a couple of reasons: 1) to give JMS users a familiar interface and 2) to provide a simple interface for reading and writing text messages (e.g. xml). Content-type: text/xml -- Content-type: application/octet-stream With that said, I'm not seeing how I can do that mapping if the transformer is provided only in the SUBSCRIBE. A client could potentially get a variety of message types from a single subscription. I think it would have to be part of the MESSAGE frame, rather than the SUBSCRIBE. SUBSCRIBE activemq-transformer: com.example.ContentTypeMapper Here are the use cases I see: s/transformer/activemq-transformer/g I like the namespace. Client-Server 1) SEND\n...\ntransformer:text (client tells server it's a text message) +1 2)
Re: STOMP and JMSType
On 6/14/06, Mittler, Nathan [EMAIL PROTECTED] wrote: -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Hiram Chirino Sent: Wednesday, June 14, 2006 12:40 PM To: activemq-dev@geronimo.apache.org Subject: Re: STOMP and JMSType On 6/14/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 13, 2006, at 3:06 PM, Nathan Mittler wrote: So it sounds like we're all in agreement on the content-type header. For text, it would be something like text and for bytes it would be application/octet-stream. So this would not be an application-level header, but would be used by my stomp client code to determine which message type to create. Content-type is application level. I was suggesting it for your use case where you want to know what to convert a bytes message into in your C++ library =) Agreed! the app in this use case is the C++ client lib. so the c++ client lib could set the content-type to control a custom transformer so that sent JMS messages are exactly as the C++ stomp lib wants them to look like. Ok, so application-level is referring to the C++ library, not the user of the library? If so that eliminates the need for another header like amq-msg-type. How do we make this become part of the stomp spec? When we do, we should define the list of valid values for it (e.g. text and bytes). So here's a link to everything that is in the spec currently: http://stomp.codehaus.org/Protocol It's a WIKI so you can edit it and improve the spec. I think that a the big missing piece in the spec is that there is no specification of how STOMP messages get mapped to JMS messages. Since this is missing, there is no provider independent way of sending JMS messages from STOMP. Since every implementation could map a STOMP message to JMS messages differently. I think we need to add a STOMP Message to JMS Message Mapping section that providers SHOULD comply with if the provider also implements a JMS interface. The great part is since this is missing, you can make this whatever you want! So I'm starting to think there are 2 main use cases: 1) I want to have portable STOMP client that work on other providers. Then you accept that you can not tightly integrate with an existing JMS network in a portable way. For example they would not be able to send and receive JMS Map messages. Since stomp does not specify what those messages would look like on the wire. This means that STOMP needs to define how a portable mapping of JMS ByteMessage and TextMessage to STOMP Messages. I think what we have today is very close to this. We may just need to formalize it a little more in a document so that other providers could implement the same STOMP to JMS mapping. Of course, this mapping has to stay simple. 2) You have a STOMP client that does not mind intimately knowing about ActiveMQ. Then it can request transformation on the the send and receives. That transformation could totally change all the STOMP rules about the headers for for the messages coming in and out. It might use the content-type to hold the JMS message type: bytes, text, object, map, etc. and other headers like jms.Type to hold the JMSType headers. Also the payload encoding could be fancier. So by default, I think it should work like case #1, if you want to use case#2, then you use the transform header options. This gives us backward compatibility but for your C++ stomp client that exposes a JMS like API use case, I would think it falls under use case #2. So right now, I'm just concerned with #1. I'd like to make the first crack at our client as STOMP vendor independent as possible ... and we're only doing text and bytes messages for the first cut. Ok great! If we're in agreement that the two use cases you've identified are two separate JIRA issues, then we can just create a second JIRA for #2, and I can go off and implement #1 in the broker. Sure! BTW, what is the timeframe for 4.1? I like to aim for 1 month, and accept slipping a month or two :) I would say it's mostly dependent on new feature progress and code stability. Also, as offshoot idea on #2, is a provider allowed able to expose additional verbs/operations to clients? Kinda like how SVN/DAV does for HTTP? If we're all in agreement with that, then it seems to make sense that the default functionality of the broker be modified to handle content- type in this way. No. activemq-transformer would provide the JMS type mapping to override the default. I suggested that you use content-type (not required by stomp) to decide if something is text or a byte stream. Transfer-encoding is also useful for this. :-) And if that's true, then it seems like this particular issue is resolved. This way, we get it into the 4.1 release with no problems. We can create another issue to do the refactoring as you've suggested ... which
Re: STOMP and JMSType
On 6/14/06, Mittler, Nathan [EMAIL PROTECTED] wrote: For map messages, on the c++ side of things we would essentially have to write a schema file for the layout of a map message. The c++ stomp client would have to be coded against that schema to properly parse the message. Unfortunately, we don't have the luxury of using a tool like XStream that uses reflection ... makes you wonder why people still code in c++! =) ;) A stomp client could just expose the XML as text; then the client application can process it however it wants - as plain text or do regex on it or use some xml parser of its choosing etc. My rambling about XStream was really focussed on the Stomp transport inside the ActiveMQ broker - i.e. the stomp - JMS bridge - rather than the clients. I'd not want to force all Stomp clients to support some kinda XStream feature. I like the rule that telnet has to be a compliant Stomp cilent :) - James --- http://radio.weblogs.com/0112098/
RE: STOMP and JMSType
Excellent! I'm breathing a sigh of relief :) -Original Message- From: James Strachan [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 14, 2006 8:55 AM To: activemq-dev@geronimo.apache.org Subject: Re: STOMP and JMSType On 6/14/06, Mittler, Nathan [EMAIL PROTECTED] wrote: For map messages, on the c++ side of things we would essentially have to write a schema file for the layout of a map message. The c++ stomp client would have to be coded against that schema to properly parse the message. Unfortunately, we don't have the luxury of using a tool like XStream that uses reflection ... makes you wonder why people still code in c++! =) ;) A stomp client could just expose the XML as text; then the client application can process it however it wants - as plain text or do regex on it or use some xml parser of its choosing etc. My rambling about XStream was really focussed on the Stomp transport inside the ActiveMQ broker - i.e. the stomp - JMS bridge - rather than the clients. I'd not want to force all Stomp clients to support some kinda XStream feature. I like the rule that telnet has to be a compliant Stomp cilent :) - James --- http://radio.weblogs.com/0112098/
Re: STOMP and JMSType
On Jun 13, 2006, at 3:06 PM, Nathan Mittler wrote: So it sounds like we're all in agreement on the content-type header. For text, it would be something like text and for bytes it would be application/octet-stream. So this would not be an application-level header, but would be used by my stomp client code to determine which message type to create. Content-type is application level. I was suggesting it for your use case where you want to know what to convert a bytes message into in your C++ library =) If we're all in agreement with that, then it seems to make sense that the default functionality of the broker be modified to handle content- type in this way. No. activemq-transformer would provide the JMS type mapping to override the default. I suggested that you use content-type (not required by stomp) to decide if something is text or a byte stream. Transfer-encoding is also useful for this. :-) And if that's true, then it seems like this particular issue is resolved. This way, we get it into the 4.1 release with no problems. We can create another issue to do the refactoring as you've suggested ... which will probably take a little more time and several conversations to get right. How does this sound? Nate On 6/13/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 13, 2006, at 1:50 PM, Nathan Mittler wrote: Could you guys point me to a place in AMQ where this sort of thing is being done? That would save me a lot of searching =) I'm viewing this problem from the client side - the Stomp C++ client that Tim Bish and I are writing currently supports text and bytes messages. Within a general Stomp client, I would suggest that switching on JMS message types is not a productive goal. Using Content-type here makes a lot more sense, I think. . It would make a lot of sense to set it for text messages going out to Stomp if there is not one already supplied. Stomp is a protocol, like HTTP or SMTP -- hardwiring a client to a specific server implementation is probably not a general solution (though is fine if it is specifically an activemq client which happens to use stomp for transport). So when I get a stomp frame in, I need to map it back to a text or bytes message. We chose to do this for a couple of reasons: 1) to give JMS users a familiar interface and 2) to provide a simple interface for reading and writing text messages (e.g. xml). Content-type: text/xml -- Content-type: application/octet-stream With that said, I'm not seeing how I can do that mapping if the transformer is provided only in the SUBSCRIBE. A client could potentially get a variety of message types from a single subscription. I think it would have to be part of the MESSAGE frame, rather than the SUBSCRIBE. SUBSCRIBE activemq-transformer: com.example.ContentTypeMapper Here are the use cases I see: s/transformer/activemq-transformer/g I like the namespace. Client-Server 1) SEND\n...\ntransformer:text (client tells server it's a text message) +1 2) SEND\n...\ntransformer:bytes (client tells server it's a bytes message) +1 3) SEND\n...\ntransformer:default (client tells server to use content-length to make determination) -1 Give it a descriptive name so that we can change the default without breaking these. 4) SEND\n...\n (no transformer specified - same as #3) +1 5) SEND\n..\ntransformer:bob (client gives server unknown transformer - use default) Return an error -- do not quietly swallow this. Server-Client 1) MESSAGE\n...\ntransformer:text (server tells client it's a text message) +1 2) MESSAGE\n...\ntransformer:bytes (server tells client it's a bytes message) +1 3) MESSAGE\n...\ntransformer:default (server tells client to use content-length to make determination) -1 same as #3 above 4) MESSAGE\n...\n (no transformer specified - same as #3) +1 5) MESSAGE\n...\ntransformer:bob (server tells client to use unknown transformer - use default) -1 same as #5 above This does highlight that we have two real transform cases, send and receive if we support CONNECT or SUBSCRIBE level transformers. We can infer the correct direct on MESSAGE and SEND, but not the others. As this would make the interface have all of two methods, I am quite happy combining it. Alternately we can have different headers on SUBSCRIBE and CONNECT -Brian
Re: STOMP and JMSType
On Jun 14, 2006, at 10:21 AM, Mittler, Nathan wrote: Ok, so application-level is referring to the C++ library, not the user of the library? If so that eliminates the need for another header like amq-msg-type. We still want the transform header for the stomp adaptor though, in order to allow people to migrate to 5.0 style (all bytes byt default) mapping. How do we make this become part of the stomp spec? When we do, we should define the list of valid values for it (e.g. text and bytes). We don't. It is how we map it in ActiveMQ, it is not part of the protocol itself. I think that having an appendix with recommended mapping is a good thing, though. Ditto an appendix on making good use of it -- Stomp Layer 2 =) So I'm starting to think there are 2 main use cases: 1) I want to have portable STOMP client that work on other providers. Then you accept that you can not tightly integrate with an existing JMS network in a portable way. For example they would not be able to send and receive JMS Map messages. Since stomp does not specify what those messages would look like on the wire. This means that STOMP needs to define how a portable mapping of JMS ByteMessage and TextMessage to STOMP Messages. I think what we have today is very close to this. We may just need to formalize it a little more in a document so that other providers could implement the same STOMP to JMS mapping. Of course, this mapping has to stay simple. Simple is good -- ask, What would Alan Kay do? =) [snip-and-rearrange] So right now, I'm just concerned with #1. I'd like to make the first crack at our client as STOMP vendor independent as possible ... and we're only doing text and bytes messages for the first cut. If you think it is important to have something specifically for text messages in the C++ client, I would switch on content-type using mime type stuff a la SMTP and HTTP. This, though, is like the magic handling of www-url-encoded form stuff in servlets -- it is a common case made more convenient in the API (which is a good thing). 2) You have a STOMP client that does not mind intimately knowing about ActiveMQ. Then it can request transformation on the the send and receives. That transformation could totally change all the STOMP rules about the headers for for the messages coming in and out. It might use the content-type to hold the JMS message type: bytes, text, object, map, etc. and other headers like jms.Type to hold the JMSType headers. Also the payload encoding could be fancier. Yes. So by default, I think it should work like case #1, if you want to use case#2, then you use the transform header options. This gives us backward compatibility but for your C++ stomp client that exposes a JMS like API use case, I would think it falls under use case #2. If we're in agreement that the two use cases you've identified are two separate JIRA issues, then we can just create a second JIRA for #2, and I can go off and implement #1 in the broker. Not sure we are all talking the same thing yet, but we are definitely getting close. -Brian
Re: STOMP and JMSType
On 6/14/06, Hiram Chirino [EMAIL PROTECTED] wrote: On 6/14/06, Nathan Mittler [EMAIL PROTECTED] wrote: A whiteboard would be really handy ... you guys aren't in NY by any chance, are you? ;) DOH I was just there last week (for 3 hours)! Want to visit Tampa Florida? HA ... give me a call after hurricane season! So what I've gathered from the previous exchange is that we need to provide a stomp-to-jms message mapping as Hiram suggested (as a stomp extension for AMQ) yep. Here are what I see as the use cases for my CMS client: 1) Stomp client sends a CMS::TextMessage, JMS client receives a TextMessage 2) Stomp client sends a CMS::BytesMessage, JMS client receives a BytesMessage 3) JMS client sends a TextMessage, Stomp client receives a CMS::TextMessage 4) JMS client sends a BytesMessage, Stomp client receives a CMS::BytesMessage Yep this is clear. 5) JMS client sends !(text || bytes message) - Stomp client error - unsupported jms message type. (One way this could be resolved is to have some sort of transformer at the broker that stuffs Object and Map messages into a bytes or text message ... perhaps as XML, like James suggested ... this, however is a different issue altogether) I would say if your CMS client wants to stay vendor neutral and not use a ActiveMQ transformation, then we can 1) give an error to the client 2) avoid consuming those types of messages by adding a selector predicate on the subscription or 3) silently consume those messages. I'm partial to #2. So here are the things I need you guys to help me fill in: So, from you previous email, I'm assuming the following questions is in regards to a standarized and portable STOMP to JMS mapping. Currently I think the way we implemented it (correct me if I'm wrong), is that if a STOMP message has content-length, then it's turned into a BytesMessage. And if it does not then its' turned into a TextMessage. And the reverse would be true when sending down to STOMP client from the broker. Are we talking about improving this so that it's more explicit? 1) What header should I use to do this mapping? 2) What are the standard values for this header and how do I make a determination when I receive a message in the C++ client of text or bytes? 3) Does this header represent some JMS-specific property that would be settable in the JMS Message interface, such as JMSMessageType? If so, we should choose another header, because the user could screw up the mapping done by the CMS library. For example if the client creates a CMS::TextMessage, this property should be internally set to that of a text message and the user should not be able to change it. I guess this all depends on what the issues are with the current way that we do thing. It seems to me that using the content-length header will allow us to switch between bytes and text just fine. But it could be we are talking about making this mapping more explicit than what we are currently doing. I think I created the issue in response to a user that wanted more explicit control over the message type that is received by JMS. Since STOMP suggests that all messages have content-length, it seems that you might want that on all of your messages, including TextMessages. But your question is valid ... I guess if we just post what happens with AMQ and when (perhaps we already do), then the users can just adjust their clients accordingly. The way we have it now should work for the CMS client - in fact it already does work with the previous version of CMS that is in trunk now. So I vote that we mark this issue as won't fix and lay this topic to rest! u with me? -- Regards, Hiram
Re: STOMP and JMSType
On 6/14/06, Nathan Mittler [EMAIL PROTECTED] wrote: On 6/14/06, Hiram Chirino [EMAIL PROTECTED] wrote: On 6/14/06, Nathan Mittler [EMAIL PROTECTED] wrote: A whiteboard would be really handy ... you guys aren't in NY by any chance, are you? ;) DOH I was just there last week (for 3 hours)! Want to visit Tampa Florida? HA ... give me a call after hurricane season! So what I've gathered from the previous exchange is that we need to provide a stomp-to-jms message mapping as Hiram suggested (as a stomp extension for AMQ) yep. Here are what I see as the use cases for my CMS client: 1) Stomp client sends a CMS::TextMessage, JMS client receives a TextMessage 2) Stomp client sends a CMS::BytesMessage, JMS client receives a BytesMessage 3) JMS client sends a TextMessage, Stomp client receives a CMS::TextMessage 4) JMS client sends a BytesMessage, Stomp client receives a CMS::BytesMessage Yep this is clear. 5) JMS client sends !(text || bytes message) - Stomp client error - unsupported jms message type. (One way this could be resolved is to have some sort of transformer at the broker that stuffs Object and Map messages into a bytes or text message ... perhaps as XML, like James suggested ... this, however is a different issue altogether) I would say if your CMS client wants to stay vendor neutral and not use a ActiveMQ transformation, then we can 1) give an error to the client 2) avoid consuming those types of messages by adding a selector predicate on the subscription or 3) silently consume those messages. I'm partial to #2. So here are the things I need you guys to help me fill in: So, from you previous email, I'm assuming the following questions is in regards to a standarized and portable STOMP to JMS mapping. Currently I think the way we implemented it (correct me if I'm wrong), is that if a STOMP message has content-length, then it's turned into a BytesMessage. And if it does not then its' turned into a TextMessage. And the reverse would be true when sending down to STOMP client from the broker. Are we talking about improving this so that it's more explicit? 1) What header should I use to do this mapping? 2) What are the standard values for this header and how do I make a determination when I receive a message in the C++ client of text or bytes? 3) Does this header represent some JMS-specific property that would be settable in the JMS Message interface, such as JMSMessageType? If so, we should choose another header, because the user could screw up the mapping done by the CMS library. For example if the client creates a CMS::TextMessage, this property should be internally set to that of a text message and the user should not be able to change it. I guess this all depends on what the issues are with the current way that we do thing. It seems to me that using the content-length header will allow us to switch between bytes and text just fine. But it could be we are talking about making this mapping more explicit than what we are currently doing. I think I created the issue in response to a user that wanted more explicit control over the message type that is received by JMS. Since STOMP suggests that all messages have content-length, it seems that you might want that on all of your messages, including TextMessages. But your question is valid ... I guess if we just post what happens with AMQ and when (perhaps we already do), then the users can just adjust their clients accordingly. The way we have it now should work for the CMS client - in fact it already does work with the previous version of CMS that is in trunk now. So I vote that we mark this issue as won't fix and lay this topic to rest! u with me? I guess that is the question. For sake of backward compatibility I would say keep it as is. But we should revisit this once the STOMP spec has a more specific Appendix on how to do stomp to jms mapping. -- Regards, Hiram -- Regards, Hiram
Re: STOMP and JMSType
I think that clears things up for me a bit - what you're proposing makes sense. I'll poke around today and see what I can come up with. Thanks, Nate On 6/12/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 12, 2006, at 4:14 PM, Nathan Mittler wrote: Agreed ... using the type header is not an option. --- From the bug report --- It isn't possible to reuse the type header (JMSType) for the purpose of sending through the information as to what type of message it is (text or bytes). So this means that we need to add another activemq extension header. I propose amq-msg-type. --- / From the bug report -- I like this a lot better, and think it would be a reasonable default rule for mapping in 4.X. I am not convinced we need this, but I much prefer it to a hardcoded transform, as this would allow for much more useful transforms (ie, aplication-aware). Although I agree conceptually, I'm thinking this might be a bit of an overkill for the task at hand. Agreed. I just hate to layer on another backwards compatibility issue beyond what we already have. By designing it to be forward compatible with an arbitrary mapping we can grow into a future solution more easily. Once we add this header we will need to support it ~forever. Right now the STOMP transport only works with bytes and text messages, and creating this transform model won't change that. I think if we one day decide to refactor the transport to accept other message types - that would be the time to make this sort of change. What if I want to switch on Content-type to decide between text or bytes? It is a common header to use, but is not part of the spec (as stomp doesn;t care, but is happy to pass it along). This makes more sense to me in terms of mapping between Stomp and JMS, but it is not compatible with switching on a specific content header. The mapping between Stomp and JMS is actually rather important to get right as it is the low level interop mapping between various platforms and Java. As such, I want to make sure we are building towards a correct solution. Aside from all this, controlling the protocol -- (semi-)protocol mapping should be a configuration thing, not a flag the client must put on every message it sends. The end goal for me is to have all messages coming in from the Stomp adaptor be bytes messages, unless someone has an overriding need for something else (quote possible). The current behavior is pretty bad as a default, but we just released 4.0 with it, so we are stuck until we make another backwards incompatible release (5.0). In 4.1 we can add the amq-msg-type header to allow people to force things to bytes (or text) but for the 5.0 end game we will need to make the mapping pluggable in order to make the upgrade path as easy as possible. if we are going to need pluggable eventually, why not do it now in order to allow people to fix the bytes/text mistake (I can say it is a mistake, I wrote it =) at the server level instead of having to add a header to every message. We have, then, three configurations which people are likely to want: 1) Current (3.2 and 4.0 compatible) one which is made more palatable by letting the client specify via the amq-msg-type. 2) Map everything to bytes, which I would like to make the default in 5.0. 3) Map everything to Text (which is what I would actually use if we had it as I convert all the bytes ones I send now into strings anyway). If we are going to have it be sufficiently pluggable to support these three, we should consider letting users provide their own. -Brian
Re: STOMP and JMSType
FWIW I'd like to have content-type header support. Couldn't we then use content-type as the standard header that any Stomp-JMS bridge would use to decide if something is a TextMessage or a BytesMessage? On 6/13/06, Nathan Mittler [EMAIL PROTECTED] wrote: I think that clears things up for me a bit - what you're proposing makes sense. I'll poke around today and see what I can come up with. Thanks, Nate On 6/12/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 12, 2006, at 4:14 PM, Nathan Mittler wrote: Agreed ... using the type header is not an option. --- From the bug report --- It isn't possible to reuse the type header (JMSType) for the purpose of sending through the information as to what type of message it is (text or bytes). So this means that we need to add another activemq extension header. I propose amq-msg-type. --- / From the bug report -- I like this a lot better, and think it would be a reasonable default rule for mapping in 4.X. I am not convinced we need this, but I much prefer it to a hardcoded transform, as this would allow for much more useful transforms (ie, aplication-aware). Although I agree conceptually, I'm thinking this might be a bit of an overkill for the task at hand. Agreed. I just hate to layer on another backwards compatibility issue beyond what we already have. By designing it to be forward compatible with an arbitrary mapping we can grow into a future solution more easily. Once we add this header we will need to support it ~forever. Right now the STOMP transport only works with bytes and text messages, and creating this transform model won't change that. I think if we one day decide to refactor the transport to accept other message types - that would be the time to make this sort of change. What if I want to switch on Content-type to decide between text or bytes? It is a common header to use, but is not part of the spec (as stomp doesn;t care, but is happy to pass it along). This makes more sense to me in terms of mapping between Stomp and JMS, but it is not compatible with switching on a specific content header. The mapping between Stomp and JMS is actually rather important to get right as it is the low level interop mapping between various platforms and Java. As such, I want to make sure we are building towards a correct solution. Aside from all this, controlling the protocol -- (semi-)protocol mapping should be a configuration thing, not a flag the client must put on every message it sends. The end goal for me is to have all messages coming in from the Stomp adaptor be bytes messages, unless someone has an overriding need for something else (quote possible). The current behavior is pretty bad as a default, but we just released 4.0 with it, so we are stuck until we make another backwards incompatible release (5.0). In 4.1 we can add the amq-msg-type header to allow people to force things to bytes (or text) but for the 5.0 end game we will need to make the mapping pluggable in order to make the upgrade path as easy as possible. if we are going to need pluggable eventually, why not do it now in order to allow people to fix the bytes/text mistake (I can say it is a mistake, I wrote it =) at the server level instead of having to add a header to every message. We have, then, three configurations which people are likely to want: 1) Current (3.2 and 4.0 compatible) one which is made more palatable by letting the client specify via the amq-msg-type. 2) Map everything to bytes, which I would like to make the default in 5.0. 3) Map everything to Text (which is what I would actually use if we had it as I convert all the bytes ones I send now into strings anyway). If we are going to have it be sufficiently pluggable to support these three, we should consider letting users provide their own. -Brian -- James --- http://radio.weblogs.com/0112098/
Re: STOMP and JMSType
I agree with all that. In a pure Stomp world, content-type is a very useful header and probably affect how the stomp client processes the message. Whatever the default rule is for Stomp - JMS I'd have thought most JMS providers would (through configuration or implementation) probably use that to make a sensible choice over Text v Bytes - even if it may break backward compatibility. (It should be trivial to configure backwards compatibility - everything is BytesMessage if folks really care) On 6/13/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 13, 2006, at 8:11 AM, Mittler, Nathan wrote: James, I think that's what we're proposing here. I proposed amq-msg-type, but I suppose content-type is just as good. I think Brian's main concern (Brian, correct me if I'm wrong) is that he'd like the mapping of stomp message to AMQ message type to be pluggable, not hard-coded. Actually, my first choice would be hard coded to bytes message, period, finished =) We cannot do this without breaking backwards compatibility, which I have been presuming we don't want to do. The most important thing, to me, when mapping from Stomp to JMS is to have it be very predictable. I don't want to ever have someone have a JMS client listening for stuff from Stomp and having to guess at the message type. As such, the default should be it is X. If they need some other mapping, I would suggest that they look at some kind of transformer service which republishes messages. If we are going to have more complicated mapping, I want to either make it super crippled: a default compatibility mode and a strong recommended 5.0 mode. The next step is to make mapping pluggable, but don't make a big deal out of it. As ben Laurie would say, a European style API. That is the external point of view. For the internal point of view, the cleanest implementation I can think of is to have an interface which takes a Send (or something like) and returns an ActiveMQMessage. I am quite strongly against using using application level metadata (which content-type is in Stomp and JMSType is in JMS) for mapping information. I am also quite against every client having to specify what to map the message to as the client should be able to be independent of the particular implementation. I think it is a very bad idea to require the client to add the mapping to every message. -Brian So if a user wanted to, they could configure the broker to turn all stomp messages into bytes messages, rather than using the default behavior of handling text and bytes messages differently. I guess I'm just not sure yet what this pluggable infrastructure should look like, as I don't think there is currently a framework in place. I already have a build with the hard-coded approach that is backward compatible with clients that don't use the amq-msg-type header. I guess I'll leave this as a question of what should be done: 1) submit what I've got (the hard-coded approach), once it's verified to be working. 2) step back and create a pluggable framework. Regards, Nate -Original Message- From: James Strachan [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 13, 2006 10:57 AM To: activemq-dev@geronimo.apache.org Subject: Re: STOMP and JMSType FWIW I'd like to have content-type header support. Couldn't we then use content-type as the standard header that any Stomp-JMS bridge would use to decide if something is a TextMessage or a BytesMessage? On 6/13/06, Nathan Mittler [EMAIL PROTECTED] wrote: I think that clears things up for me a bit - what you're proposing makes sense. I'll poke around today and see what I can come up with. Thanks, Nate On 6/12/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 12, 2006, at 4:14 PM, Nathan Mittler wrote: Agreed ... using the type header is not an option. --- From the bug report --- It isn't possible to reuse the type header (JMSType) for the purpose of sending through the information as to what type of message it is (text or bytes). So this means that we need to add another activemq extension header. I propose amq-msg-type. --- / From the bug report -- I like this a lot better, and think it would be a reasonable default rule for mapping in 4.X. I am not convinced we need this, but I much prefer it to a hardcoded transform, as this would allow for much more useful transforms (ie, aplication-aware). Although I agree conceptually, I'm thinking this might be a bit of an overkill for the task at hand. Agreed. I just hate to layer on another backwards compatibility issue beyond what we already have. By designing it to be forward compatible with an arbitrary mapping we can grow into a future solution more easily. Once we add this header we will need to support it ~forever. Right now the STOMP transport only works with bytes and text messages, and creating this transform model won't change that. I think if we one day decide to refactor the transport
RE: STOMP and JMSType
Sounds good to me. +1 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Hiram Chirino Sent: Tuesday, June 13, 2006 1:45 PM To: activemq-dev@geronimo.apache.org Subject: Re: STOMP and JMSType On 6/13/06, Mittler, Nathan [EMAIL PROTECTED] wrote: Just to make clear what the proposed new hard-coded logic does: 1) If there is no content-length, it's a text message (same as before) 2) else, if there is no message type, it's a bytes message (same as before) 3) else use amq-msg-type to determine the message type So this logic is backward-compatible with the existing logic. If the client does not specify amq-msg-type, the behavior will be the same as it is now. The STOMP protocol doesn't define a mapping to a JMS system, so as far as predictability goes, the users just need to follow our logic for the mapping. They wouldn't be able to change the mapping, like they would with a pluggable framework, but they shouldn't need to. They just code their clients against the mapping that we publish online and they'll be up and running. I'm getting the feeling that the main reason for leaning toward the pluggable framework is to support a BytesMessage-only paradigm. In a STOMP-only world this probably makes sense. You're just and receiving stuff. But when you're bridging between Stomp and JMS, I'm not sure why a client would ever want this restriction. It seems like if you want BytesMessages to come out the other end, then you just follow our mapping logic to make that happen, which really is as simple as including content-length (which you would have to include for a true bytes message anyway) and leave off amq-msg-type (which you would by default). Even if you were to implement a pluggable framework, you would still need to have some sort of application metadata like the amq-msg-type header. Assuming our framework is comprised of a bunch of message transformers that go between raw bytes and ActiveMQMessages, the default transformer for the STOMP transport would do this mapping the way I've proposed above. So you wouldn't be eliminating the need for the amq-msg-type header, you'd just be pushing around the logic that uses it. So I guess I'm still not seeing the bang for the buck ... it seems like the hard-coded solution works for all cases that we've defined so far. I'd rather just get this functionality in because I think it's useful and people have been asking for it. I'm all for continuing to discuss the pluggable framework, but it seems that that is a separate issue from what I'm trying to do here (...and probably literally a separate JIRA issue). I agree. But instead calling the header 'amq-msg-type' (which does not explicitly convey that a transformation is being used), I'd like it to be called 'activemq-transformation' set to a class name of the transformation (or something that maps to a classname, like we do for our service discovery). and this would be header that is not carried with the message, it's a header that controls the send and subscribe operations. Regards, Nate -Original Message- From: Brian McCallister [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 13, 2006 12:20 PM To: activemq-dev@geronimo.apache.org Subject: Re: STOMP and JMSType On Jun 13, 2006, at 8:11 AM, Mittler, Nathan wrote: James, I think that's what we're proposing here. I proposed amq-msg-type, but I suppose content-type is just as good. I think Brian's main concern (Brian, correct me if I'm wrong) is that he'd like the mapping of stomp message to AMQ message type to be pluggable, not hard-coded. Actually, my first choice would be hard coded to bytes message, period, finished =) We cannot do this without breaking backwards compatibility, which I have been presuming we don't want to do. The most important thing, to me, when mapping from Stomp to JMS is to have it be very predictable. I don't want to ever have someone have a JMS client listening for stuff from Stomp and having to guess at the message type. As such, the default should be it is X. If they need some other mapping, I would suggest that they look at some kind of transformer service which republishes messages. If we are going to have more complicated mapping, I want to either make it super crippled: a default compatibility mode and a strong recommended 5.0 mode. The next step is to make mapping pluggable, but don't make a big deal out of it. As ben Laurie would say, a European style API. That is the external point of view. For the internal point of view, the cleanest implementation I can think of is to have an interface which takes a Send (or something like) and returns an ActiveMQMessage. I am quite strongly against using using application level metadata (which content-type is in Stomp and JMSType is in JMS) for mapping information. I am also quite against every client having to specify what to map the message
Re: STOMP and JMSType
I would suggest that we use a weak form of specification encoding. ActiveMQ already uses pseudo-urls in a lot of places, and named transforms trump class names, usually: yeah! URI style configuration rocks! -Brian -- Regards, Hiram
Re: STOMP and JMSType
So it sounds like we're all in agreement on the content-type header. For text, it would be something like text and for bytes it would be application/octet-stream. So this would not be an application-level header, but would be used by my stomp client code to determine which message type to create. If we're all in agreement with that, then it seems to make sense that the default functionality of the broker be modified to handle content-type in this way. And if that's true, then it seems like this particular issue is resolved. This way, we get it into the 4.1 release with no problems. We can create another issue to do the refactoring as you've suggested ... which will probably take a little more time and several conversations to get right. How does this sound? Nate On 6/13/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 13, 2006, at 1:50 PM, Nathan Mittler wrote: Could you guys point me to a place in AMQ where this sort of thing is being done? That would save me a lot of searching =) I'm viewing this problem from the client side - the Stomp C++ client that Tim Bish and I are writing currently supports text and bytes messages. Within a general Stomp client, I would suggest that switching on JMS message types is not a productive goal. Using Content-type here makes a lot more sense, I think. . It would make a lot of sense to set it for text messages going out to Stomp if there is not one already supplied. Stomp is a protocol, like HTTP or SMTP -- hardwiring a client to a specific server implementation is probably not a general solution (though is fine if it is specifically an activemq client which happens to use stomp for transport). So when I get a stomp frame in, I need to map it back to a text or bytes message. We chose to do this for a couple of reasons: 1) to give JMS users a familiar interface and 2) to provide a simple interface for reading and writing text messages (e.g. xml). Content-type: text/xml -- Content-type: application/octet-stream With that said, I'm not seeing how I can do that mapping if the transformer is provided only in the SUBSCRIBE. A client could potentially get a variety of message types from a single subscription. I think it would have to be part of the MESSAGE frame, rather than the SUBSCRIBE. SUBSCRIBE activemq-transformer: com.example.ContentTypeMapper Here are the use cases I see: s/transformer/activemq-transformer/g I like the namespace. Client-Server 1) SEND\n...\ntransformer:text (client tells server it's a text message) +1 2) SEND\n...\ntransformer:bytes (client tells server it's a bytes message) +1 3) SEND\n...\ntransformer:default (client tells server to use content-length to make determination) -1 Give it a descriptive name so that we can change the default without breaking these. 4) SEND\n...\n (no transformer specified - same as #3) +1 5) SEND\n..\ntransformer:bob (client gives server unknown transformer - use default) Return an error -- do not quietly swallow this. Server-Client 1) MESSAGE\n...\ntransformer:text (server tells client it's a text message) +1 2) MESSAGE\n...\ntransformer:bytes (server tells client it's a bytes message) +1 3) MESSAGE\n...\ntransformer:default (server tells client to use content-length to make determination) -1 same as #3 above 4) MESSAGE\n...\n (no transformer specified - same as #3) +1 5) MESSAGE\n...\ntransformer:bob (server tells client to use unknown transformer - use default) -1 same as #5 above This does highlight that we have two real transform cases, send and receive if we support CONNECT or SUBSCRIBE level transformers. We can infer the correct direct on MESSAGE and SEND, but not the others. As this would make the interface have all of two methods, I am quite happy combining it. Alternately we can have different headers on SUBSCRIBE and CONNECT -Brian
Re: STOMP and JMSType
BTW I confess to skim reading this thread a little so appologies if I missed some subtleties along the way :) So I guess there's 2 things. (i) Whats the default mapping of stomp content-type - JMS Message that most stomp-jms providers should support by default. (ii) Allowing folks to customize the mapping of Stomp headers - JMS Messages. I'm all for both; I wanna nail down (i) and implement it ASAP. Option (ii) we can tinker with over time as its probably gonna be a pluggable strategy we can tweak over time. e.g. I can imagine use cases where when we see application/xml or text/xml or application/soap or whatnot to using something like JAXB/SDO to make an ObjectMessage containing the marshalled body. On 6/13/06, Mittler, Nathan [EMAIL PROTECTED] wrote: James, I think that's what we're proposing here. I proposed amq-msg-type, but I suppose content-type is just as good. I think Brian's main concern (Brian, correct me if I'm wrong) is that he'd like the mapping of stomp message to AMQ message type to be pluggable, not hard-coded. So if a user wanted to, they could configure the broker to turn all stomp messages into bytes messages, rather than using the default behavior of handling text and bytes messages differently. I guess I'm just not sure yet what this pluggable infrastructure should look like, as I don't think there is currently a framework in place. I already have a build with the hard-coded approach that is backward compatible with clients that don't use the amq-msg-type header. I guess I'll leave this as a question of what should be done: 1) submit what I've got (the hard-coded approach), once it's verified to be working. 2) step back and create a pluggable framework. Regards, Nate -Original Message- From: James Strachan [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 13, 2006 10:57 AM To: activemq-dev@geronimo.apache.org Subject: Re: STOMP and JMSType FWIW I'd like to have content-type header support. Couldn't we then use content-type as the standard header that any Stomp-JMS bridge would use to decide if something is a TextMessage or a BytesMessage? On 6/13/06, Nathan Mittler [EMAIL PROTECTED] wrote: I think that clears things up for me a bit - what you're proposing makes sense. I'll poke around today and see what I can come up with. Thanks, Nate On 6/12/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 12, 2006, at 4:14 PM, Nathan Mittler wrote: Agreed ... using the type header is not an option. --- From the bug report --- It isn't possible to reuse the type header (JMSType) for the purpose of sending through the information as to what type of message it is (text or bytes). So this means that we need to add another activemq extension header. I propose amq-msg-type. --- / From the bug report -- I like this a lot better, and think it would be a reasonable default rule for mapping in 4.X. I am not convinced we need this, but I much prefer it to a hardcoded transform, as this would allow for much more useful transforms (ie, aplication-aware). Although I agree conceptually, I'm thinking this might be a bit of an overkill for the task at hand. Agreed. I just hate to layer on another backwards compatibility issue beyond what we already have. By designing it to be forward compatible with an arbitrary mapping we can grow into a future solution more easily. Once we add this header we will need to support it ~forever. Right now the STOMP transport only works with bytes and text messages, and creating this transform model won't change that. I think if we one day decide to refactor the transport to accept other message types - that would be the time to make this sort of change. What if I want to switch on Content-type to decide between text or bytes? It is a common header to use, but is not part of the spec (as stomp doesn;t care, but is happy to pass it along). This makes more sense to me in terms of mapping between Stomp and JMS, but it is not compatible with switching on a specific content header. The mapping between Stomp and JMS is actually rather important to get right as it is the low level interop mapping between various platforms and Java. As such, I want to make sure we are building towards a correct solution. Aside from all this, controlling the protocol -- (semi-)protocol mapping should be a configuration thing, not a flag the client must put on every message it sends. The end goal for me is to have all messages coming in from the Stomp adaptor be bytes messages, unless someone has an overriding need for something else (quote possible). The current behavior is pretty bad as a default, but we just released 4.0 with it, so we are stuck until we make another backwards incompatible release (5.0). In 4.1 we can add the amq-msg-type header to allow people to force things to bytes (or text) but for the 5.0 end game we will need
RE: STOMP and JMSType
Just to make clear what the proposed new hard-coded logic does: 1) If there is no content-length, it's a text message (same as before) 2) else, if there is no message type, it's a bytes message (same as before) 3) else use amq-msg-type to determine the message type So this logic is backward-compatible with the existing logic. If the client does not specify amq-msg-type, the behavior will be the same as it is now. The STOMP protocol doesn't define a mapping to a JMS system, so as far as predictability goes, the users just need to follow our logic for the mapping. They wouldn't be able to change the mapping, like they would with a pluggable framework, but they shouldn't need to. They just code their clients against the mapping that we publish online and they'll be up and running. I'm getting the feeling that the main reason for leaning toward the pluggable framework is to support a BytesMessage-only paradigm. In a STOMP-only world this probably makes sense. You're just and receiving stuff. But when you're bridging between Stomp and JMS, I'm not sure why a client would ever want this restriction. It seems like if you want BytesMessages to come out the other end, then you just follow our mapping logic to make that happen, which really is as simple as including content-length (which you would have to include for a true bytes message anyway) and leave off amq-msg-type (which you would by default). Even if you were to implement a pluggable framework, you would still need to have some sort of application metadata like the amq-msg-type header. Assuming our framework is comprised of a bunch of message transformers that go between raw bytes and ActiveMQMessages, the default transformer for the STOMP transport would do this mapping the way I've proposed above. So you wouldn't be eliminating the need for the amq-msg-type header, you'd just be pushing around the logic that uses it. So I guess I'm still not seeing the bang for the buck ... it seems like the hard-coded solution works for all cases that we've defined so far. I'd rather just get this functionality in because I think it's useful and people have been asking for it. I'm all for continuing to discuss the pluggable framework, but it seems that that is a separate issue from what I'm trying to do here (...and probably literally a separate JIRA issue). Regards, Nate -Original Message- From: Brian McCallister [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 13, 2006 12:20 PM To: activemq-dev@geronimo.apache.org Subject: Re: STOMP and JMSType On Jun 13, 2006, at 8:11 AM, Mittler, Nathan wrote: James, I think that's what we're proposing here. I proposed amq-msg-type, but I suppose content-type is just as good. I think Brian's main concern (Brian, correct me if I'm wrong) is that he'd like the mapping of stomp message to AMQ message type to be pluggable, not hard-coded. Actually, my first choice would be hard coded to bytes message, period, finished =) We cannot do this without breaking backwards compatibility, which I have been presuming we don't want to do. The most important thing, to me, when mapping from Stomp to JMS is to have it be very predictable. I don't want to ever have someone have a JMS client listening for stuff from Stomp and having to guess at the message type. As such, the default should be it is X. If they need some other mapping, I would suggest that they look at some kind of transformer service which republishes messages. If we are going to have more complicated mapping, I want to either make it super crippled: a default compatibility mode and a strong recommended 5.0 mode. The next step is to make mapping pluggable, but don't make a big deal out of it. As ben Laurie would say, a European style API. That is the external point of view. For the internal point of view, the cleanest implementation I can think of is to have an interface which takes a Send (or something like) and returns an ActiveMQMessage. I am quite strongly against using using application level metadata (which content-type is in Stomp and JMSType is in JMS) for mapping information. I am also quite against every client having to specify what to map the message to as the client should be able to be independent of the particular implementation. I think it is a very bad idea to require the client to add the mapping to every message. -Brian
Re: STOMP and JMSType
The think your views are a bit STOMP point of view centric. bytes messages work fine when both end assume you are just moving around bytes messages.. and it's true everything can eventually converted down to a byte[]. If you look at it from a JMS point of view, how will a STOMP client deal with an existing JMS network? What if a JMS MapMessage or an ObjectMessage is sent to a stomp client? The question is how will that be encoded to bytes[].. And what if the STOMP client needs to talk to an App that expect JMS MapMessages or ObjectMessages?? Yes I guess using a ESB style transformer in the middle would solve the issue, but I think it adds complexity in the overall solution. It would be easier if the STOMP client could send a Map or Object message directly even if it is an ActiveMQ extension that allows it to do that. Perhaps we add a header on send: activemq-tranformation: org.a.a.transform.StompTextToMapMessage And on subscribe we add another header to covert on the way in. activemq-tranformation: org.a.a.transform.MapMessageToStompText And if those are just class names, then the users could implement thier own transformation implementations and just add them to the broker classpath. Regards, Hiram On 6/13/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 13, 2006, at 8:11 AM, Mittler, Nathan wrote: James, I think that's what we're proposing here. I proposed amq-msg-type, but I suppose content-type is just as good. I think Brian's main concern (Brian, correct me if I'm wrong) is that he'd like the mapping of stomp message to AMQ message type to be pluggable, not hard-coded. Actually, my first choice would be hard coded to bytes message, period, finished =) We cannot do this without breaking backwards compatibility, which I have been presuming we don't want to do. The most important thing, to me, when mapping from Stomp to JMS is to have it be very predictable. I don't want to ever have someone have a JMS client listening for stuff from Stomp and having to guess at the message type. As such, the default should be it is X. If they need some other mapping, I would suggest that they look at some kind of transformer service which republishes messages. If we are going to have more complicated mapping, I want to either make it super crippled: a default compatibility mode and a strong recommended 5.0 mode. The next step is to make mapping pluggable, but don't make a big deal out of it. As ben Laurie would say, a European style API. That is the external point of view. For the internal point of view, the cleanest implementation I can think of is to have an interface which takes a Send (or something like) and returns an ActiveMQMessage. I am quite strongly against using using application level metadata (which content-type is in Stomp and JMSType is in JMS) for mapping information. I am also quite against every client having to specify what to map the message to as the client should be able to be independent of the particular implementation. I think it is a very bad idea to require the client to add the mapping to every message. -Brian So if a user wanted to, they could configure the broker to turn all stomp messages into bytes messages, rather than using the default behavior of handling text and bytes messages differently. I guess I'm just not sure yet what this pluggable infrastructure should look like, as I don't think there is currently a framework in place. I already have a build with the hard-coded approach that is backward compatible with clients that don't use the amq-msg-type header. I guess I'll leave this as a question of what should be done: 1) submit what I've got (the hard-coded approach), once it's verified to be working. 2) step back and create a pluggable framework. Regards, Nate -Original Message- From: James Strachan [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 13, 2006 10:57 AM To: activemq-dev@geronimo.apache.org Subject: Re: STOMP and JMSType FWIW I'd like to have content-type header support. Couldn't we then use content-type as the standard header that any Stomp-JMS bridge would use to decide if something is a TextMessage or a BytesMessage? On 6/13/06, Nathan Mittler [EMAIL PROTECTED] wrote: I think that clears things up for me a bit - what you're proposing makes sense. I'll poke around today and see what I can come up with. Thanks, Nate On 6/12/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 12, 2006, at 4:14 PM, Nathan Mittler wrote: Agreed ... using the type header is not an option. --- From the bug report --- It isn't possible to reuse the type header (JMSType) for the purpose of sending through the information as to what type of message it is (text or bytes). So this means that we need to add another activemq extension header. I propose amq-msg-type. --- / From the bug report -- I like this a lot better, and think it would be a reasonable default rule for mapping in 4.X. I am
Re: STOMP and JMSType
On 6/13/06, Mittler, Nathan [EMAIL PROTECTED] wrote: Just to make clear what the proposed new hard-coded logic does: 1) If there is no content-length, it's a text message (same as before) 2) else, if there is no message type, it's a bytes message (same as before) 3) else use amq-msg-type to determine the message type So this logic is backward-compatible with the existing logic. If the client does not specify amq-msg-type, the behavior will be the same as it is now. The STOMP protocol doesn't define a mapping to a JMS system, so as far as predictability goes, the users just need to follow our logic for the mapping. They wouldn't be able to change the mapping, like they would with a pluggable framework, but they shouldn't need to. They just code their clients against the mapping that we publish online and they'll be up and running. I'm getting the feeling that the main reason for leaning toward the pluggable framework is to support a BytesMessage-only paradigm. In a STOMP-only world this probably makes sense. You're just and receiving stuff. But when you're bridging between Stomp and JMS, I'm not sure why a client would ever want this restriction. It seems like if you want BytesMessages to come out the other end, then you just follow our mapping logic to make that happen, which really is as simple as including content-length (which you would have to include for a true bytes message anyway) and leave off amq-msg-type (which you would by default). Even if you were to implement a pluggable framework, you would still need to have some sort of application metadata like the amq-msg-type header. Assuming our framework is comprised of a bunch of message transformers that go between raw bytes and ActiveMQMessages, the default transformer for the STOMP transport would do this mapping the way I've proposed above. So you wouldn't be eliminating the need for the amq-msg-type header, you'd just be pushing around the logic that uses it. So I guess I'm still not seeing the bang for the buck ... it seems like the hard-coded solution works for all cases that we've defined so far. I'd rather just get this functionality in because I think it's useful and people have been asking for it. I'm all for continuing to discuss the pluggable framework, but it seems that that is a separate issue from what I'm trying to do here (...and probably literally a separate JIRA issue). I agree. But instead calling the header 'amq-msg-type' (which does not explicitly convey that a transformation is being used), I'd like it to be called 'activemq-transformation' set to a class name of the transformation (or something that maps to a classname, like we do for our service discovery). and this would be header that is not carried with the message, it's a header that controls the send and subscribe operations. Regards, Nate -Original Message- From: Brian McCallister [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 13, 2006 12:20 PM To: activemq-dev@geronimo.apache.org Subject: Re: STOMP and JMSType On Jun 13, 2006, at 8:11 AM, Mittler, Nathan wrote: James, I think that's what we're proposing here. I proposed amq-msg-type, but I suppose content-type is just as good. I think Brian's main concern (Brian, correct me if I'm wrong) is that he'd like the mapping of stomp message to AMQ message type to be pluggable, not hard-coded. Actually, my first choice would be hard coded to bytes message, period, finished =) We cannot do this without breaking backwards compatibility, which I have been presuming we don't want to do. The most important thing, to me, when mapping from Stomp to JMS is to have it be very predictable. I don't want to ever have someone have a JMS client listening for stuff from Stomp and having to guess at the message type. As such, the default should be it is X. If they need some other mapping, I would suggest that they look at some kind of transformer service which republishes messages. If we are going to have more complicated mapping, I want to either make it super crippled: a default compatibility mode and a strong recommended 5.0 mode. The next step is to make mapping pluggable, but don't make a big deal out of it. As ben Laurie would say, a European style API. That is the external point of view. For the internal point of view, the cleanest implementation I can think of is to have an interface which takes a Send (or something like) and returns an ActiveMQMessage. I am quite strongly against using using application level metadata (which content-type is in Stomp and JMSType is in JMS) for mapping information. I am also quite against every client having to specify what to map the message to as the client should be able to be independent of the particular implementation. I think it is a very bad idea to require the client to add the mapping to every message. -Brian -- Regards, Hiram
Re: STOMP and JMSType
On 6/13/06, Brian McCallister [EMAIL PROTECTED] wrote: On Jun 13, 2006, at 10:36 AM, Hiram Chirino wrote: The think your views are a bit STOMP point of view centric. bytes messages work fine when both end assume you are just moving around bytes messages.. and it's true everything can eventually converted down to a byte[]. Yes, and no. I want to see ActiveMQ become the de facto standard for messaging, period. Part of that is supporting non-JMS really well. So, yes, I am looking at it from a non-Java-centric (but allowing for Java via JMS) point of view =) If you look at it from a JMS point of view, how will a STOMP client deal with an existing JMS network? What if a JMS MapMessage or an ObjectMessage is sent to a stomp client? The question is how will that be encoded to bytes[].. And what if the STOMP client needs to talk to an App that expect JMS MapMessages or ObjectMessages?? This seems, to me, to be a strong argument for a pluggable framework for message transformation. Going from JMS to Stomp for non Text/ Bytes will be quite application specific, I think. In that case having a single solution applied to the whole messaging system is pretty ugly, period. If you are bridging from JMS to stomp, and using things other than bytes or text, it is going to be kind of ugly. Yes I guess using a ESB style transformer in the middle would solve the issue, but I think it adds complexity in the overall solution. It would be easier if the STOMP client could send a Map or Object message directly even if it is an ActiveMQ extension that allows it to do that. For any messaging system used for multiple applications, I think that mapping as a republishing service Perhaps we add a header on send: activemq-tranformation: org.a.a.transform.StompTextToMapMessage I like this name better. Stomp is verbose anyway =) Being able to send a transformation specification is powerful, but kind of scary. I *really* think this should be a configurable option on the server, rather than have to have the client know what is happening on the other end. Allowing the client to specify is good, requiring the client to specify is bad. So if the the client does not specifies a transformation, the 'default' transformation will be used, which is hopefully backward compatible with what we are doing today in 4.0 and 3.x. And perhaps we allow the 'default' transformation the be configurable on the server side so that the default can be changed. But perhaps this would over complicate things as clients would expect a default behaviour which we have now allowed the a server admin to change. And on subscribe we add another header to covert on the way in. activemq-tranformation: org.a.a.transform.MapMessageToStompText That is a very interesting idea. On subscribe you can specify a transformer to apply to whole session by default. A step up from per message, but I still think this should be able to be specified on the server config as well. And if those are just class names, then the users could implement thier own transformation implementations and just add them to the broker classpath. I would suggest that we use a weak form of specification encoding. ActiveMQ already uses pseudo-urls in a lot of places, and named transforms trump class names, usually: activemq-transformer: class:org.example.WombatTransformer activemq-transformer: named:wombat-transformer inline: ruby:it.gsub /kangaroos/i, wombats inline is not a good idea =) -Brian -- Regards, Hiram
Re: STOMP and JMSType
Could you guys point me to a place in AMQ where this sort of thing is being done? That would save me a lot of searching =) I'm viewing this problem from the client side - the Stomp C++ client that Tim Bish and I are writing currently supports text and bytes messages. So when I get a stomp frame in, I need to map it back to a text or bytes message. We chose to do this for a couple of reasons: 1) to give JMS users a familiar interface and 2) to provide a simple interface for reading and writing text messages (e.g. xml). With that said, I'm not seeing how I can do that mapping if the transformer is provided only in the SUBSCRIBE. A client could potentially get a variety of message types from a single subscription. I think it would have to be part of the MESSAGE frame, rather than the SUBSCRIBE. Here are the use cases I see: Client-Server 1) SEND\n...\ntransformer:text (client tells server it's a text message) 2) SEND\n...\ntransformer:bytes (client tells server it's a bytes message) 3) SEND\n...\ntransformer:default (client tells server to use content-length to make determination) 4) SEND\n...\n (no transformer specified - same as #3) 5) SEND\n..\ntransformer:bob (client gives server unknown transformer - use default) Server-Client 1) MESSAGE\n...\ntransformer:text (server tells client it's a text message) 2) MESSAGE\n...\ntransformer:bytes (server tells client it's a bytes message) 3) MESSAGE\n...\ntransformer:default (server tells client to use content-length to make determination) 4) MESSAGE\n...\n (no transformer specified - same as #3) 5) MESSAGE\n...\ntransformer:bob (server tells client to use unknown transformer - use default) Does this make sense, or am I missing something? Thanks, Nate On 6/13/06, Hiram Chirino [EMAIL PROTECTED] wrote: I would suggest that we use a weak form of specification encoding. ActiveMQ already uses pseudo-urls in a lot of places, and named transforms trump class names, usually: yeah! URI style configuration rocks! -Brian -- Regards, Hiram
Re: STOMP and JMSType
On Jun 13, 2006, at 1:50 PM, Nathan Mittler wrote: Could you guys point me to a place in AMQ where this sort of thing is being done? That would save me a lot of searching =) I'm viewing this problem from the client side - the Stomp C++ client that Tim Bish and I are writing currently supports text and bytes messages. Within a general Stomp client, I would suggest that switching on JMS message types is not a productive goal. Using Content-type here makes a lot more sense, I think. . It would make a lot of sense to set it for text messages going out to Stomp if there is not one already supplied. Stomp is a protocol, like HTTP or SMTP -- hardwiring a client to a specific server implementation is probably not a general solution (though is fine if it is specifically an activemq client which happens to use stomp for transport). So when I get a stomp frame in, I need to map it back to a text or bytes message. We chose to do this for a couple of reasons: 1) to give JMS users a familiar interface and 2) to provide a simple interface for reading and writing text messages (e.g. xml). Content-type: text/xml -- Content-type: application/octet-stream With that said, I'm not seeing how I can do that mapping if the transformer is provided only in the SUBSCRIBE. A client could potentially get a variety of message types from a single subscription. I think it would have to be part of the MESSAGE frame, rather than the SUBSCRIBE. SUBSCRIBE activemq-transformer: com.example.ContentTypeMapper Here are the use cases I see: s/transformer/activemq-transformer/g I like the namespace. Client-Server 1) SEND\n...\ntransformer:text (client tells server it's a text message) +1 2) SEND\n...\ntransformer:bytes (client tells server it's a bytes message) +1 3) SEND\n...\ntransformer:default (client tells server to use content-length to make determination) -1 Give it a descriptive name so that we can change the default without breaking these. 4) SEND\n...\n (no transformer specified - same as #3) +1 5) SEND\n..\ntransformer:bob (client gives server unknown transformer - use default) Return an error -- do not quietly swallow this. Server-Client 1) MESSAGE\n...\ntransformer:text (server tells client it's a text message) +1 2) MESSAGE\n...\ntransformer:bytes (server tells client it's a bytes message) +1 3) MESSAGE\n...\ntransformer:default (server tells client to use content-length to make determination) -1 same as #3 above 4) MESSAGE\n...\n (no transformer specified - same as #3) +1 5) MESSAGE\n...\ntransformer:bob (server tells client to use unknown transformer - use default) -1 same as #5 above This does highlight that we have two real transform cases, send and receive if we support CONNECT or SUBSCRIBE level transformers. We can infer the correct direct on MESSAGE and SEND, but not the others. As this would make the interface have all of two methods, I am quite happy combining it. Alternately we can have different headers on SUBSCRIBE and CONNECT -Brian
Re: STOMP and JMSType
On 6/13/06, Nathan Mittler [EMAIL PROTECTED] wrote: Could you guys point me to a place in AMQ where this sort of thing is being done? That would save me a lot of searching =) Its not really done anywhere else :-( I'm viewing this problem from the client side - the Stomp C++ client that Tim Bish and I are writing currently supports text and bytes messages. So when I get a stomp frame in, I need to map it back to a text or bytes message. We chose to do this for a couple of reasons: 1) to give JMS users a familiar interface and 2) to provide a simple interface for reading and writing text messages (e.g. xml). Sure. that would be handy. But you could also eventualy support Map messages too right? With that said, I'm not seeing how I can do that mapping if the transformer is provided only in the SUBSCRIBE. A client could potentially get a variety of message types from a single subscription. I think it would have to be part of the MESSAGE frame, rather than the SUBSCRIBE. Well, when the publisher is a pure jms client, he wont be providing a header for the transform type right? What I was thinking is that when the client says SUBSCRIBE with x transform, it basically sets up a contract that given all the JMS message types a well defined transformation to STOMP frames will be done. It could be that a future transform will take a Map messages and turn it into XML and put that in a text stomp frame. But a different transformation might take a Map message and do a binary encoding of the key value pairs. Either way the client will know what to expect because he requested it on the SUBSCRIBE. In addition, a smarter transformer could enhance the subscription's selector so that it does not receive message types that it cannot handle. That way if the client says it can only receive text messages, it only subscribes to text messages. Here are the use cases I see: Client-Server 1) SEND\n...\ntransformer:text (client tells server it's a text message) 2) SEND\n...\ntransformer:bytes (client tells server it's a bytes message) 3) SEND\n...\ntransformer:default (client tells server to use content-length to make determination) 4) SEND\n...\n (no transformer specified - same as #3) 5) SEND\n..\ntransformer:bob (client gives server unknown transformer - use default) and I also see stuff similar to: 6) SEND\n...\ntransformer:xml-to-map (client tells server it's a xml stomp frame that need to get converted to a jms MAP message) Server-Client 1) MESSAGE\n...\ntransformer:text (server tells client it's a text message) 2) MESSAGE\n...\ntransformer:bytes (server tells client it's a bytes message) 3) MESSAGE\n...\ntransformer:default (server tells client to use content-length to make determination) 4) MESSAGE\n...\n (no transformer specified - same as #3) 5) MESSAGE\n...\ntransformer:bob (server tells client to use unknown transformer - use default) Sure, but that's just content type information. I do agree we need to set this to something. But it's separate from doing transforms to map from all the JMS message types to the 2 basic stomp types text and bytes. Does this make sense, or am I missing something? I think we are starting to converge now. I guess the biggest issues with content type like fields is that there are so many of them. I personally would map content-type - JMSType since I think they were meant to hold app specific descriptions of the payload. We may want to have the additional header you were proposing to hold information about the STOMP frame - JMS Message type kind of information. It gets fuzzy here. Thanks, Nate On 6/13/06, Hiram Chirino [EMAIL PROTECTED] wrote: I would suggest that we use a weak form of specification encoding. ActiveMQ already uses pseudo-urls in a lot of places, and named transforms trump class names, usually: yeah! URI style configuration rocks! -Brian -- Regards, Hiram -- Regards, Hiram
Re: STOMP and JMSType
JMSType is a reserved header in JMS, for use at the application
level. I think what you are proposing is more accurately an ActiveMQ
specific transform header. I think this type of transform should
either be a real, arbitrary, pluggable, transform mechanism, or
should not be done.
I would much prefer to *always* use a bytes message, but this is
backwards incompatible so cannot be done in 4.X.
I would propose that instead of overloading JMSType, if we think we
need to have a transform/mapping mechanism we base it on an active-mq
specific header, and make it something like:
STOMPMessageTransformer {
public ActiveMQMessage transform
(SomeRepresentationOfTheSendFrameIncludingHeaders frame) {
...
}
}
I am not convinced we need this, but I much prefer it to a hardcoded
transform, as this would allow for much more useful transforms (ie,
aplication-aware).
I think that, properly, this should be done by a service on the
messaging bus though, NOT in a protocol handler.
-Brian
On Jun 12, 2006, at 12:59 PM, Mittler, Nathan wrote:
I'm working on fixing the way the STOMP transport determines Text and
Bytes messages for issue AMQ-739. Previously we keyed off of the
content-length header - if it was there, it's a bytes message, and
otherwise it's a text message.
Since all STOMP messages can have content-length, we need to use
JMSType
to distinguish in these cases. To do this, we need to define standard
JMSType values for Text and Bytes messages. I have a build that uses
text and bytes as the standard values for the type header.
On the
broker side, the logic in Send.java looks like this ...
** BEGIN CODE **
// Assume the message is a bytes message.
Boolean isBytesMessage = true;
// If the message does not contain a content length,
// we have to assume it's a text message - first zero
// we encounter denotes the end of the frame.
If( !headers.containsKey(Stomp.Headers.CONTENT_LENGTH) ){
isBytesMessage = false;
}
// There is a content length specified,
// now use JMSType to determine the message type (default to bytes if
none specified)
else if( headers.containsKey( Stomp.Headers.Send.TYPE ) ){
isBytesMessage =
(headers.getProperty(Stomp.Headers.Send.TYPE) ==
Stomp.Headers.TypeValues.BYTES);
}
if( isBytesMessage ){
// create a bytes message.
}else{
// create a text message.
}
** END CODE ***
Any objections?
Regards,
Nate
Re: STOMP and JMSType
On Jun 12, 2006, at 4:14 PM, Nathan Mittler wrote: Agreed ... using the type header is not an option. --- From the bug report --- It isn't possible to reuse the type header (JMSType) for the purpose of sending through the information as to what type of message it is (text or bytes). So this means that we need to add another activemq extension header. I propose amq-msg-type. --- / From the bug report -- I like this a lot better, and think it would be a reasonable default rule for mapping in 4.X. I am not convinced we need this, but I much prefer it to a hardcoded transform, as this would allow for much more useful transforms (ie, aplication-aware). Although I agree conceptually, I'm thinking this might be a bit of an overkill for the task at hand. Agreed. I just hate to layer on another backwards compatibility issue beyond what we already have. By designing it to be forward compatible with an arbitrary mapping we can grow into a future solution more easily. Once we add this header we will need to support it ~forever. Right now the STOMP transport only works with bytes and text messages, and creating this transform model won't change that. I think if we one day decide to refactor the transport to accept other message types - that would be the time to make this sort of change. What if I want to switch on Content-type to decide between text or bytes? It is a common header to use, but is not part of the spec (as stomp doesn;t care, but is happy to pass it along). This makes more sense to me in terms of mapping between Stomp and JMS, but it is not compatible with switching on a specific content header. The mapping between Stomp and JMS is actually rather important to get right as it is the low level interop mapping between various platforms and Java. As such, I want to make sure we are building towards a correct solution. Aside from all this, controlling the protocol -- (semi-)protocol mapping should be a configuration thing, not a flag the client must put on every message it sends. The end goal for me is to have all messages coming in from the Stomp adaptor be bytes messages, unless someone has an overriding need for something else (quote possible). The current behavior is pretty bad as a default, but we just released 4.0 with it, so we are stuck until we make another backwards incompatible release (5.0). In 4.1 we can add the amq-msg-type header to allow people to force things to bytes (or text) but for the 5.0 end game we will need to make the mapping pluggable in order to make the upgrade path as easy as possible. if we are going to need pluggable eventually, why not do it now in order to allow people to fix the bytes/text mistake (I can say it is a mistake, I wrote it =) at the server level instead of having to add a header to every message. We have, then, three configurations which people are likely to want: 1) Current (3.2 and 4.0 compatible) one which is made more palatable by letting the client specify via the amq-msg-type. 2) Map everything to bytes, which I would like to make the default in 5.0. 3) Map everything to Text (which is what I would actually use if we had it as I convert all the bytes ones I send now into strings anyway). If we are going to have it be sufficiently pluggable to support these three, we should consider letting users provide their own. -Brian
