On 28/02/14 16:04, Rob Godfrey wrote:
Hi Fraser,
wd07 took out all the language about the form of bodies in repsonses I
think :-)
Jakub has already pointed out that the locales still used an illegal list
(doh!).
As to lists in the bodies... The JMS Mapping that Robbie is working on will
define exactly how these get passed back. Given the nature of the data an
ObjectMessage is probably going to be the only sane way of doing so
(there's no guarantee that any of the data within the inner lists is going
to be valid for JMS Map messages... e.g. one of the "attributes" might
itself have a list value).
Yippee!!! You might recall this, but one of my - many :-) - soap-boxes
relates to issues of encoding AMQP Maps let alone AMQP Lists into JMS
Message types. I've attached a thread from a year past January that was
precipitated by changes to List encoding in Qpid 0.20 in that I said:
"Possibly the main reason using MapMessage is flawed though is because
the JMS Spec says:
"A |MapMessage| object is used to send a set of name-value pairs. The
names are |String| objects, and the values are *primitive data types in
the Java programming language*", that's right "primitive data types". To
be fair it then goes on to include byte[] in the list and it *does*
include a getObject() and setObject(), but the JavaDoc explicitly says
"This method works only for the objectified primitive object types
(|Integer|, |Double|, |Long| ...), |String| objects, and byte arrays. "
If Robbie is looking at this it might be worth you guys looking through
the whole thread "JMS List support & QMF2 - was Re: [ANNOUNCE] Apache
Qpid 0.20 released" as it happens it was vaguely controversial and TBH I
don't think at the time I was winning the ObjectMessage argument
although what I was saying felt correct (to me).
I'm kind of glad you guys appear to be reaching a similar conclusion but
given the discussions around it back then it's probably worthwhile
giving the others involved an opportunity to reiterate their position.
One slight AMQP 1.0 related niggle with ObjectMessages I *think* that
the AMQP 1.0 spec states a preference for not setting content-type
("When using an application-data section with a section code other than
data, content-type
SHOULD NOT be set."). I mention this because Gordon mentioned "the
preferred method of sending maps and lists is through an AmqpValue
section with the type encoded in the bytes making up that section" (see
second attached mail).
Technically that would mean that according to the spec. for Map or List
encoding the content-type should not be set. In itself it's probably not
a big deal but I think that the implication is that one would need to
use the slightly ugly (IMHO) instanceof in order to determine the actual
type of the ObjectMessage.
This is one of those things that is likely to be "exciting" when trying
to design applications that can interact with a range of broker
versions. I already got bitten between 0.18 and 0.20 with the change of
List representation then (and IIRC something changed between 0.8 and
0.10 too). It's fairly easy to get things working consistently on trunk
and forget that many people have a mixed economy of versions - though
retaining backwards compatibility is hard I'd agree :-/
I agree the attributes as a list on the message body inbound and the first
row on the response message is a bit of a hack... I could personally live
with a Map of { "attributes" -> <list of attributes>, "data" -> <list of
lists> }, or having attributes be a comma separated String in the
application-properties.
I personally think the Map suggestion is cleaner. I did wonder about the
comma separated string and it's *fairly* easy to "parse" though unless
you explicitly disallow whitespace the regex to split the values becomes
more fiddly and aside from that it feels kind of hacky too.
I'll try to put out a WD08 shortly at least addressing the illegal locales
thing.
BTW WD07 can be downloaded here:
https://www.oasis-open.org/committees/document.php?document_id=52286&wg_abbrev=amqp
-- Rob
Cheers. Would you be able to post links to the working drafts when they
get uploaded TBH I wasn't aware of WD07, the last link I saw was WD06
(or is there a top level link that shows all the WDs to date)
Regards,
Frase
--- Begin Message ---
As a follow up to my posting below.
QMF2 is indeed broken by Java changes in Qpid 0.20 and my hunch around
somebody messing with how Content-Type amqp/list is encoded and decoded
does indeed look to be "the smoking gun".
I added some debug code to my getObjects() and I was hitherto receiving
a BytesMessage, which did need some decoding to retrieve the List, but
now it is receiving a MapMessage.
I've only started looking, but doing:
MapMessage m = (MapMessage)response;
for (Enumeration e = m.getMapNames() ; e.hasMoreElements() ;)
{
String key = (String)e.nextElement();
System.out.println("key = " + key);
}
indicates (for the "broker" object) "key = 0"
Which is suggesting that Lists now seem to be encoded in MapMessages
with keys that are indices :'( .
To be fair Rajith Attapattu did post on this subject at the end of August:
http://qpid.2158936.n2.nabble.com/JMS-ListMessage-support-tp7581434.html
and perhaps I *should* have taken more notice, however it was a single
posting and pretty inconclusive, so I don't thing that it was
unreasonable for me to park it in "things to look at once the QMF stuff
is released" given where I was trying to focus my spare time.
IMHO this is *exactly* the sort of thing that acts as a "/quod erat
demonstrandum/" to Gordon Sim's posting on "mailing lists and fragmented
communication", 'cause I'd have got a *whole lot more* animated about
the subject if I'd have thought that this stuff was going to actually
make it in to a release (see below!!).....
Have I just missed where the release notes communicated this, or is this
a change that hasn't been communicated to "users" (aside from Rajith's
posting)??
I have to say that IMHO whilst the BytesMessage stuff (and the
associated decoding) "wasn't exactly ideal" I think that the way
MapMessage has been used here using String keys as indices feels pretty
hideous too :-(
There was some talk too in Rajith's post around "Make the list message
accessible via the MapMessage and StreamMessage interfaces" I'm not sure
if this has been done, but TBH I'm not sure that I like StreamMessage
any better!!
At the risk of coming over a bit anally retentive.......Here's what I
should have taken the time to respond to
http://qpid.2158936.n2.nabble.com/JMS-ListMessage-support-tp7581434.html
with:
I don't actually like MapMessage a whole lot more than any of the above.
I think that even for exposing amqp/map its use is flawed for (at least)
two reasons:
1) MapMessage is frankly a hideous interface that should have been
kicked into touch years ago by Sun. The particular axe I've got to grind
with it is that there's this thing called java.util.Map which has been
around since Java 1.2, so it's spectacularly unhelpful to have a
different interface for Maps transported via JMS. Among other things It
really bugs me having to use Enumerations and not being able to use foreach.
It's even more irksome becase QMF often comprises Maps of Maps, so the
top level Map ends up being a MapMessage and the rest end up being
java.util.Map. It's not at all elegant and frankly I ended up writing a
"getMap(final Message message)" method that populates a java.util.Map
with the messageProperties and Map values so I'd get at least some
consistency.
2) Possibly the main reason using MapMessage is flawed though is because
the JMS Spec says:
"A |MapMessage| object is used to send a set of name-value pairs. The
names are |String| objects, and the values are *primitive data types in
the Java programming language*", that's right "primitive data types". To
be fair it then goes on to include byte[] in the list and it *does*
include a getObject() and setObject(), but the JavaDoc explicitly says
"This method works only for the objectified primitive object types
(|Integer|, |Double|, |Long| ...), |String| objects, and byte arrays. ".
So the bottom line is then that encoding java.util.Map and
java.util.List into javax.jms.MapMessage (which are clearly necessary
for QMF plus a whole load of other things) *explicitly* breaks the JMS
specification (as well as having a really hideous interface).
As for StreamMessage that says:
"A |StreamMessage| object is used to send a stream of primitive types in
the Java programming language" and the writeObject() method JavaDoc says
"This method works only for the objectified primitive object types
(|Integer|, |Double|, |Long| ...), |String| objects, and byte arrays. ".
So again explicitly primitive types.
I've said the following a couple of times on the Qpid Users mailing
list, but nobody has yet bitten, hopefully this issue might start the
discussion again....
IMHO the *only* correct way to expose amqp/map or amqp/list in JMS in a
way that doesn't break the JMS specification is to use
javax.jms.ObjectMessage.
ObjectMessage says:
"
An |ObjectMessage| object is used to send a message that contains a
serializable object in the Java programming language ("Java object"). It
inherits from the |Message| interface and adds a body containing a
single reference to an object. Only |Serializable| Java objects can be
used.
If a collection of Java objects must be sent, one of the |Collection|
classes provided since JDK 1.2 can be used.
"
The last paragraph is particularly interesting and is suggesting to me
that this is exactly what is being implicitly suggested by Sun/Oracle as
the preferred way of passing Collections of various sorts via JMS.
Now to be fair things were perhaps more awkward in older versions of
Qpid when the AMQP Content-Type wasn't getting exposed in JMS, but even
without being able to use the Content-Type to avoid speculating casting
and instanceof it still seems preferable. The Content-Type however can
add value and make the whole thing completely consistent and well
specified.
Sorry that this has gone on somewhat, and I apologise for not
contributing these thoughts when Rajith originally mentioned this (in
retrospect it would probably have saved me quite some work).
I hope that I've managed to convince the community that:
a) javax.jms.ObjectMessage is the "one true" JMS specification compliant
way of exposing amqp/map and amqp/list
b) that there's not a huge gulf between "developers" and "users" and
some of the choices can have big impacts particularly where "messing
with interfaces" in a not massively well controlled manner is concerned.
Food for thought???
I'll make a start getting the QMF2 stuff working with the Qpid 0.20
changes. Now that I know what I'm up against it's not especially
difficult, but Lists are used in a few places so I'll probably spend a
bit of time trying to make sure I've covered it all off and trying to
ensure I've abstracted sufficiently from the underlying JMS Message type
so that it won't be too hard to accommodate ObjectMessage if (when???)
things move to that.
And Rajith, if any of the stuff above comes across as me having a go at
you it's *really* not my intention. I hope it doesn't come across like
that, but emails can come across in ways not intended so I want to make
that clear up front! I really should have put all of this stuff in a
response to your post, I'm afraid I was just trying to keep focus on my
GUI. A flawed decision on my part as it turns out :-D
I'll keep you posted.
Frase
On 26/01/13 11:32, Fraser Adams wrote:
Gripes so far...... I'm afraid it looks like 0.20 has broken QMF on Java.
I've built the main qpid-all.jar and pointed my CLASSPATH at it and
unfortunately "fings is well broke, innit?" :-(
It's lower level than the GUI stuff as none of the QMF2 stuff is
working and my QpidConfig port reports "No broker QmfConsoleData
returned". My initial suspicion is still that somebody has messed
around with how AMQP list messages are deserialised in Java, but I'll
need to dig around a bit :-(
On a more minor (and I'd assume unrelated) note I see the following on
every Java Client I've fired up so far:
log4j:WARN No appenders could be found for logger
(org.apache.qpid.jndi.PropertiesFileInitialContextFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
for more info.
Odd that it's just that one class
I haven't built the C++ broker/client yet and the Java glitch seems to
concur with what Gordon saw last week (it looks like a change between
0.18 and 0.20 as when he used 0.18 jars and pointed at a 0.20 broker
things worked fine) so Java looks like the place to go digging.
I'll keep you posted.
Regards,
Frase.
On 23/01/13 15:30, Justin Ross wrote:
The Apache Qpid community is pleased to announce the immediate
availability of Apache Qpid 0.20.
Apache Qpid (http://qpid.apache.org) is a cross-platform enterprise
messaging solution which implements the Advanced Message Queuing
Protocol (AMQP, http://www.amqp.org). It provides brokers written in
C++ and Java, along with clients for C++, Java JMS, .Net, Python,
Ruby, and Perl.
Qpid 0.20 is available from our website:
http://qpid.apache.org/download.html
The 0.20 release includes many bug fixes and improvements. We
recommend that all users upgrade. A comprehensive list of changes in
the 0.20 release is available here:
http://qpid.apache.org/release_notes_0.20.html
Thanks to all the users and contributors who have helped to improve
Apache Qpid.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
--- End Message ---
--- Begin Message ---
In AMQP 0-10 message content was essentially always treated as raw
data, with the content-type indicating how those bytes were to be
interpreted. For map- and list- messages we used a content-type of
amqp/map and amqp/list respectively.
In AMQP 1.0, though you can of course still send raw data (as a Data
section) with a content-type specified in the properties, the preferred
method of sending maps and lists is through an AmqpValue section with
the type encoded in the bytes making up that section.
I've been working on support for map- and list- messages over 1.0
through the c++ qpid::messaging API and have an approach ready for some
feedback[1].
Over 0-10, the approach used was to have encode/decode functions that
work with a Variant::Map or Variant::List. The application was
responsible for encoding outgoing messages before sending them and
checking the content-type on received messages and decoding them if
appropriate.
Now that the content-type cannot be relied on to indicate the type of
the message (indeed the specification encourages it *not* to be used,
relying only on the type encoding in the AmqpValue section), a different
approach is necessary.
I've added a method to Message to get the content as a Variant. This can
be used for a whole range of different types of data including binary or
maps and lists. All the existing methods for accessing content remain of
course and in the case of structured data, they will return the raw bytes.
With this approach, the application doesn't directly encode or decode
the data itself, but simply reads or manipulates the Variant typed
content. E.g instead of:
Message message;
Variant::Map content;
content["a"] = "bcd";
encode(content, message);
sender.send(message);
a map message would be sent like this:
Message message;
message.content() = Variant::Map();
message.content()["a"] = "bcd";
sender.send(message);
and instead of:
Message message = receiver.receive();
if (message.getContentType() == "amqp/map") {
Variant::Map content;
decode(message, content);
//process content:
std::string x = content["a"];
} else {
//handle unexpected format
}
receiving a map message would look like this:
Message message = receiver.receive();
if (message.content().getType() == VAR_MAP) {
//process content:
std::string x = message.content().asMap()["a"];
} else {
//handle unexpected format
}
The old approach will of course continue to work for 0-10 without
modification, but in order to support the same pattern over both
protocols I have modified the 0-10 path also. For sending, if you don't
touch Message::content() then it will behave just like it did before.
For receiving, map- and list- messages will by default be decoded
automatically when fetched (and made available via Message::content()).
This can if desired be disabled via a connection option; it doesn't
break anything but if your application already explicitly tests the
content type and decodes then its doing redundant work.
For 1.0 you can also of course continue to send raw bytes with an
associated content-type if desired as well. I haven't actually exposed
1.0 based encode and decode routines for this purpose though.
Any feedback on this as an approach is welcome. I'd be especially
excited to hear from anyone who thinks they might want to send AMQP 1.0
encoded structured content. A patch is available for those who want to
see the gory details[1].
--Gordon.
[1] https://reviews.apache.org/r/13328/
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
--- End Message ---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
