There are a number of changes desired for federation for the M4 release:
1) Intelligent forwarding of messages based on actual bindings at the
destination.
In the current code, message forwarding is done based on
statically configured bindings to the inter-broker queues. To get
messages to consumers who have recently added bindings to their
local exchanges requires that either:
a) The federation bindings be general enough to handle any
possible new binding. This means that messages are forwarded
regardless of whether there is an eligible consumer to receive
it, wasting bandwidth.
b) The federation bindings somehow be manually changed to
accomodate the new destination bindings.
It is desired that federated brokers forward only those messages
for which there are bindings at the destination.
2) Automatic configuration of the anti-looping (tag/exclude) mechanism.
The anti-looping mechanism is hard to configure correctly and is
arguably a detail of the brokers' implementation. It would be
better if anti-looping was applied correctly without any input
from the broker administrator.
=========
I'll cover (2) first as it is a building block for (1).
The anti-looping mechanism prevents messages from being received more
than once on any given broker. This is important because most useful
federation topologies, especially topics and fanouts, involve loops.
Without anti-looping, a single message can cycle around the topology
indefinitely, being delivered to the same consumers repeatedly.
Anti-looping is implemented with a "tag and exclude" feature of
queues. The inter-broker queue is configured with two strings: the
tag and the exclusion tag. The queue marks all queued messages with
the tag (i.e. appends the tag to a list of tags) and drops all
messages that contain the exclusion tag.
A working convention for tags is as follows:
- Each broker is assigned a unique tag.
- Each federation queue uses the local (source) broker's tag and
excludes the tag for the subscribing (dest) broker.
Here's the proposal for M4:
1) Make the local tag a configuration item for the broker (vhost
actually). This can be supplied in the command line or may be set
by a remote management console. The tag is stored persistently and
need be set only once.
2) Add the tag to the server-properties and client-properties fields
in the connection.start-ok and connection.start messages
respectively. Name this property "qpid.federation_tag".
3) When federation routes are set up and their queues configured, if
tag and exclude values are not supplied in the setup method
(bridge), use the locally configured tag and the tag advertised by
the federation peer when setting up the queue.
=========
Here's the proposal for (1). I'm calling this "dynamic routing"
because it involves dynamically creating and deleting bindings on the
federation queues.
When a federation route is configured, a subscription is created on
the source broker with the "dest" field set to the name of the
destination exchange. In a "pull" route, this subscription is
initiated by the destination broker. The federation code that manages
this subscription can track the local bindings on the dest exchange
and propagate the union of the binding keys to the source exchange.
I propose that dynamic routing not be supported for "push" routes.
It is theoretically possible for the derivative bindings to be
"coalesced" or summarized into a smaller number of bindings for a
topic exchange. I have no plans to attempt this at this time.
It doesn't take much thought to realize that there is a looping issue
for dynamic bindings much like there is with messages. The solution,
of course, is also similar in that bind commands have an argument
field that can carry a tag list. Here are the rules:
1) If a local binding is untagged (i.e. it was created by a local
consumer), tag the bind request with only the local broker tag
before propagating it to any source brokers.
2) If a binding is declared and it is tagged with the local broker
tag, do not apply the binding (silently drop it).
3) If the set of tags changes on an existing binding, re-propagate it
to all sources.
When a new route is established (because it was just added or because
the remote broker just became reachable), the existing bindings of the
dest exchange must be iterated over for propagation.
There is a problem for which I don't currently have a solution. The
exchange.unbind command does not have an arguments field so
propagation of unbinds is problematic. One possibility (admittedly
ugly) is to use a field in the exchange.bind arguments to indicate
that a bind command is to be interpreted as an unbind for the purposes
of propagation.
-Ted
