Attached are some notes on the current status of the AMQP 1.0 support
for qpid::messaging and qpidd, describing how it is currently
implemented and highlighting some of the missing features and changes
with respect to addressing options. As ever, I am open to any feedback,
questions, criticisms or alternate opinions...
--Gordon.
AMQP 1.0 support for the qpid::messaging API
--------------------------------------------
The guiding principle has been to allow applications written to the
qpid::messaging API to speak AMQP 1.0 in a clear and natural way, to
avoid tying its use to any particular broker. The 0-10 support will of
course remain unaltered.
The API is itself fairly simple. It is in the address syntax and
specifically the more detailed options that much of the complexity of
the mapping lies.
Reply-To addresses and temporary queues
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There has been one minor change to the way the API itself works over
1.0. This does not affect existing 0-10 use however. The change
involves the creation of temporary queues (or topics), for retrieving
replies in a request-response pattern for example.
Over 0-10, the Address will convert a node name starting with a '#'
character by inserting a UUID. This works well for 0-10 where the name
is chose by clients and must be unique. This transformation of the
name is done when constructing an Address from a single address string
(rather than from its constituent parts). The modified name could then
be accessed via Address::getName().
Over 1.0 however the name for such nodes is determined by the
server. In this case the name assigned needs to be communicated back
to the application when the attach succeeds. To handle that a new
accessor - getAddress() - has been added to both Sender and Receiver.
In order to keep backward compatibility for 0-10, the Address
constructor still does the transformation, but applications that want
to be able to switch to 1.0 should use these new accessors to obtain
the correct address for setting reply-to on any request messages they
send. (This new approach will work for both 0-10 and 1.0).
Connections, Session and Links
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The protocol used is selected at runtime via the 'protocol' connection
property. The recognised values are 'amqp1.0' and 'amqp0-10'. AMQP
0-10 is still the default and the 1.0 support is only available if the
required module is loaded. There is no failover support for 1.0
connections yet[1].
The SASL negotiation is optional in AMQP 1.0. If no SASL layer is
desired, the sasl_mechanisms connection option can be set to NONE.
AMQP 1.0 can be used over SSL, however the messaging client does not
at this stage use an AMQP negotiated security layer for that
prupose. Peers must expect SSL on the port being used (either
exclusively or by being able to detect an SSL header).
Transactional sessions are not yet supported[2].
The creation of senders or receivers results in the attaching of a
link to the peer. The details of the attach, in particular the source
and/or target, are controlled through the address string.
Addresses
~~~~~~~~~
The name specified in the address supplied when creating a sender or
receiver is used to set the address of the target or source
respectively.
If the subject is specified for a sender it is used the default
subject for messages sent without an explicit subject set.
If the subject is specified for a receiver it is interpreted as a
filter on the set of messages of interest. If it includes a wildcard
(i.e. a '*' or a '#') it is sent as a legacy-amqp-topic-binding, if
not it is sent as a legacy-amqp-direct-binding.
When the name of the address is (or starts with) '#', the dynamic flag
is set on the corresponding source or target and the
dynamic-node-properties are populated based on the node
properties. Note that when the dynamic flag is set the address should
not be specified. However due to PROTON-277[3], I have to set the
address to something in order to work at all against another proton-c
based peer, such as qpidd (so I set it to '.'). This can be resolved
as soon as the proton bug is fixed.
As mentioned above in discussing the changes around reply-to
addresses, AMQP 1.0 doesn't allow on demand creation of nodes with a
client specified name. However, I have defined a special extension
capability for the c++ broker that will allow 'create' behaviour that
is similar to that supported over 0-10. That is, it will create a node
with the name specified by the client if it does not already exist. I
see this as a temporary measure to help transition situations that
rely on create policy at present. It is non-standard however and the
recommendation would be to look for ways to avoid relying on
that. Hopefully at some point a standard management mechanism will
provide better alternatives.
If the addressed node is to be created on demand - either through use
of '#' as the name, or through the create policy - the node properties
are sent as dynamic-node-properties on the source or target. These can
be specified in a nested map within the node. Additionally the durable
and type properties in the node map itself are sent if
specified. There is at present also a translation from the 0-10 style
x-declare in the node whereby all the fields specified in it are
included as if they had been listed in properties, the arguments map
is flattened into this.
The x-bindings property is not currently supported for AMQP 1.0 in
nodes or links. This has really been a question of priorities rather
than ruling out any mapping. Though I don't think a generic binding
solution is appropriate for the 1.0 implementation, I'm eager to here
of use cases that would be affected here and see how best to address
them.
The delete policy is not supported over 1.0 at all. There isn't really
any obvious way to map this. It was never terribly well thought
through even for AMQP 0-10 in my view. I believe what is more
important is expanding/revising the scope of nodes.
The 1.0 specification defines the following policies in connection
with nodes created in response to establishing the link:
delete-on-close, delete-on-no-links, delete-on-no-messages or
delete-on-no-links-or-messages. At present the first of these is
assumed, support for the others will be added in future.
There is some support for the assert option, but it is not entirely
equivalent to the 0-10 based mechanism. Over AMQP 1.0 it uses the
source or target capabilities. The client will set the capabilities it
desires, the broker (or other peer) will set the capabilities it can
offer and if the assert option is on, then the client will ensure all
the capabilities it requested are indeed supported.
The specific capabilities are yet to be standardised. The capabilities
sent by the client can be controlled through a nested list within the
node map. Note that capabilities are simply strings (symbols in 1.0 to
be precise), not name value pairs.
If durable is set in the node properties, then a capability of
'durable' will be requested, meaning the node will not lose messages
if its volatile memory is lost, and requested if the durable property
within the node map is present in the address.
If the type is set, then that will also be passed a requested
capability e.g. 'queue' meaning the node supports queue-like
characteristics (stores messages until consumers claim them and
allocates messages between competing consumers), 'topic' meaning the
node supports classic pub-sub characteristics.
The 'browse' mode is not yet supported due to PROTON-139[4]
The Message class and the AMQP 1.0 message format
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The message-id, correlation-id, user-id, subject, reply-to and
content-type fields in the properties section of a 1.0 message can all
be set or retreived via accessors of the same name on the Message
instance. Likewise for the durable, priority and ttl fields in the
header section.
An AMQP 1.0 message has a delivery-count field (within the header
section) for which there is no direct accessor yet on the Message
class. However if the value is greater than 1, then the
Message::getRedelivered() method will return true. Note: this is
incorrect and will be fixed[5]. If Message::setRedelivered() is called
with a value of true, then the delivery count will be set to 1, else
it will be set to 0.
The application-properties section of a 1.0 message is made available
via the properties map of the Message class. Likewuse on sending a
message the properties map is used to poplulate the
application-properties section.
There are other fields defined in the AMQP 1.0 message format that as
yet do not have direct accessors on the Message class. These will be
made available on received messages via special keys into the
properties map.
The format for the keys in general x-amqp-<field-name>. The keys
currently in use are: x-amqp-first-acquirer and x-amqp-delivery-count
for the header section, and x-amqp-to, x-amqp-absolute-expiry-time,
x-amqp-creation-time, x-amqp-group-id, x-amqp-qroup-sequence and
x-amqp-reply-to-group-id for the properties section.
In addition the delivery- and message- annotations sections are made
available via a nested maps with key x-amqp-delivery-annotations and
x-amqp-message-annotations respectively.
At present none of these special keys are recognised when sending
messages, in which case they will be transmitted as application
properties. This will be rectified shortly[6] and indeed the set of
direct accessors will likely be expanded.
AMQP 1.0 support in qpidd
-------------------------
To enable 1.0 support in qpidd, the amqp module must be loaded. This
allows the broker to recognise the 1.0 protocol header alongside the
0-10 one.
The broker can accept connections with or without and underlying SASL
security layer as defined by the 1.0 specification. However if
authentication is turned on -- i.e. auth=yes -- then a SASL security
layer MUST be used.
There is as yet no AMQP 1.0 specific integration with the ACL. This
means for example that you cannot prevent an authenticated user for
subscribing to a particular queue, or limit the number of 1.0 based
connections opened up by such a user. This will be rectified in a
future release[7].
The broker allows links in both directions to be attached to queues or
exchanges. The address in the source or target is resolved by checking
whether it matches the name of a queue or an exchange. If there exists
a queue and an exchange of the same name, the queue is used but a
warning is logged.
If the node is an exchange, then an outgoing link (i.e. messages to
travel out from broker) will cause a temporary, link-scoped queue to
be created on the broker and bound to the exchange.
Outgoing links may have a filter set on their source. The filters
currently supported by the broker are 'legacy-amqp-direct-binding',
'legacy-amqp-topic-binding' and 'selector-filter' as defined in the
registered extensions.
When the node is an exchange the 'legacy-amqp-direct-binding',
'legacy-amqp-topic-binding' filters are implemented as a binding (this
only works as expected for direct and topic exchanges). When the node
is a queue, the subscription will skip over messages that do not match
that subject (i.e the subject filter works as expected with queues
over 1.0, whereas it was ignored over 0-10). At present the selector
filter is ignored if the source is an exchange, but that will be
resolved shortly[8].
If the dynamic flag is set on the source or target, then the
dynamic-node-properties are inspected to determine the characteristics
of the node to be created. The properties that are recognised are
essentially the same as accepted via the create qmf
method. Specifically, the 0-10 defined options durable, auto-delete,
alternate-exchange, exchange-type and then any qpidd specific options
(e.g. qpid.max-count).
The supported-dist-modes property as defined by the 1.0 specfication
is used to determine whether a queue or exchange is desired (where the
create method uses the 'type' property). If 'move' is specified a
queue will be created, if 'copy' is specified an exchange will be
created. If this property is not set, then a queue is assumed.
Messages sent over AMQP 0-10 will be converted by the broker for
sending over AMQP 1.0 and vice versa.
The message-id, correlation-id, userid, content-type and
content-encoding all map fairly simply in both driections between the
properties section in 1.0 and the message-properties in an 0-10
header. Note however that in 0-10 a message-id must be a UUID, and in
translating to 0-10 from 1.0 this field will be skipped unless it is a
valid UUID.
Likewise the priority directly between the field in the header section
of a 1.0 message and the corresponding field in the
delivery-properties of an 0-10 message. The durable header in a 1.0
message is equivalent to the delivery-mode in the delivery-properties
of an 0-10 message, with a value of true in the former being
equivalent to a value of 2 in the latter and a value of false in the
former equivalent to 1 in the latter.
When converting from 0-10 to 1.0, the reply-to will be simply the
routing-key if the exchange is not set, or if it is the reply-to
address for 1.0 will be composed of the exchange and if specified the
routing key (separated by a forward slash). The reply-to is not yet
converted from 1.0 to 0-10.
When converting from 0-10 to 1.0, if the 0-10 message has a non-empty
destination, then the subject field in the properties of the 1.0
message is set to the value of the routing-key from the
message-properties of the 0-10 message. In the reverse direction, the
subject field of the properties section of the 1.0 message is used to
populate the routing-key in the message-properties of the 0-10
message.
The destination of a 0-10 message is used to populate the 'to' field
of the properties section when converting to 1.0, but the reverse
translation is not done (as the destination for messages sent out by
the broker is controlled by the subscription in 0-10).
The application-properties section of a 1.0 message is converted to
the application-headers field in the message-properties of an 0-10
message and vice versa.
No translation of the message body is done at this point[9].
The broker recognises particular capabilities for source and
targets. It will only include a given capability in the attach it
sends if that has been 'requested' in the attach sent by the peer to
trigger the establishment of the link. This gives the peer some means
of ensuring their expectations will be met.
The 'durable' capability will be added if the queue or exchange
refered to by the source or target is durable. The 'queue' capability
will be added if the source or target references a queue. The 'topic'
capability will be added if the source or target references an
exchange. If the source or target references a queue or a direct
exchange the 'legacy-amqp-direct-binding' will be added. If it
references a queue or a topic exchange, 'legacy-amqp-topic-binding'
will be added.
The 'create-on-demand' capability is currently offered as a temporary
extension offering some continuity from the commonly used 'create'
policy in the messaging client. If set in the client and the named
node does not exist, it will be created using the
dynamic-node-properties as when the dynamic flag is set (see above).
[1] https://issues.apache.org/jira/browse/QPID-4708
[2] https://issues.apache.org/jira/browse/QPID-4710
[3] https://issues.apache.org/jira/browse/QPID-4668
[4] https://issues.apache.org/jira/browse/PROTON-277
[5] https://issues.apache.org/jira/browse/PROTON-139
[6] https://issues.apache.org/jira/browse/QPID-4707
[7] https://issues.apache.org/jira/browse/QPID-4712
[8] https://issues.apache.org/jira/browse/QPID-4706
[9] https://issues.apache.org/jira/browse/QPID-4711
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
