This is an automated email from the ASF dual-hosted git repository. jbertram pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/artemis.git
commit 357084eb4023fd15164879c700b3ca11b3d7dc51 Author: Timothy Bish <[email protected]> AuthorDate: Wed Dec 10 16:36:00 2025 -0500 ARTEMIS-5797 Updated docs for AMQP federation Add more detailed documentation for AMQP federation convering more of the configuration options and example configurations. --- docs/user-manual/amqp-address-federation.adoc | 136 ++++++++++++ docs/user-manual/amqp-broker-connections.adoc | 96 +-------- .../amqp-federation-configuration-glossary.adoc | 240 +++++++++++++++++++++ docs/user-manual/amqp-queue-federation.adoc | 107 +++++++++ .../amqp-federation-address-bi-directional.png | Bin 0 -> 50127 bytes .../images/amqp-federation-address-concept.png | Bin 0 -> 78785 bytes .../images/amqp-federation-address-hub-spoke.png | Bin 0 -> 89448 bytes .../images/amqp-federation-address-mesh.png | Bin 0 -> 80926 bytes .../images/amqp-federation-address-ring.png | Bin 0 -> 83891 bytes .../amqp-federation-queue-bi-directional.png | Bin 0 -> 49608 bytes .../images/amqp-federation-queue-concept.png | Bin 0 -> 71224 bytes .../images/amqp-federation-queue-simple-lb.png | Bin 0 -> 41395 bytes .../images/amqp-federation-queue-with-hop.png | Bin 0 -> 49248 bytes docs/user-manual/index.adoc | 4 +- 14 files changed, 496 insertions(+), 87 deletions(-) diff --git a/docs/user-manual/amqp-address-federation.adoc b/docs/user-manual/amqp-address-federation.adoc new file mode 100644 index 0000000000..d666700eed --- /dev/null +++ b/docs/user-manual/amqp-address-federation.adoc @@ -0,0 +1,136 @@ += Address Federation +:idprefix: +:idseparator: - +:docinfo: shared + +Address federation can be thought of as full multicast over the connected brokers. +Every message sent to address on `Broker-1` will be delivered to every queue bound to that address on that broker, but also will be delivered to the matching address on `Broker-2`, `Broker-3` ... `Broker-N` and all the queues bound to that address. + +Address federation creates consumers that read messages from the matching address on the remote peer when demand is detected on a local address that matches a specific federation address policy. On the remote peer a queue binding is created on the federated address which will store and forward messages sent to that address back to the initiating broker. Once a message arrives at the federation address consumer on the initiating broker it is then published into the local address which rou [...] + +Address federation can be configured to operate in a single direction between peers or can configured to operate in both directions between peers allowing message to flow in both directions delivering messages to consumers on either broker that were sent to the matching address on the opposing broker instance. + +image::images/amqp-federation-address-concept.png[] + +== Common Address Federation Topologies + +The sections that follow discuss some common address federation topologies and some configuration considerations specific to those topologies. + +=== Two way Address Federation + +This is a common case where consumers on either broker should receive messages sent from producers both on the local broker and on the remote broker. Messages only cross the federation connection if a consumer exists on the remote address otherwise messages do not leave the local broker. + +image::images/amqp-federation-address-bi-directional.png[] + +This configuration can be setup by adding a federation policy on both broker instances that configure the address to federate and a connection URI pointing to the opposing broker. + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> + <federation> + <local-address-policy name="example-local-address-policy"> + <include address-match="a.federated-address.#" /> + </local-address-policy> + </federation> + </amqp-connection> +</broker-connections> +---- + +It is also possible to configure this on only one of the two brokers using a remote address federation policy in the broker connection configuration. + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> + <federation> + <local-address-policy name="example-local-address-policy"> + <include address-match="a.federated-address.#" /> + </local-address-policy> + <remote-address-policy name="example-remote-address-policy"> + <include address-match="a.federated-address.#" /> + </remote-address-policy> + </federation> + </amqp-connection> +</broker-connections> +---- + +In either of these two configurations the federation implementation will protect itself from looping messages so unlike other implementations you do not need to configure a `max-hops` setting. + +=== Three Broker Mesh + +An expansion of the previous configuration is a three broker mesh where each broker creates an address federation for the same address on each of the other two brokers when local demand is present. In this scenario any message published on an address on one of the brokers will be sent to consumers on that address that are active on the other two brokers. + +image::images/amqp-federation-address-mesh.png[] + +Because this configuration now creates a loop with more than two brokers the configuration must add a `max-hops` value to ensure that the messages sent to one of the brokers does not propagate beyond the two closest neighbors in the loop otherwise the messages would loop forever amongst the three brokers. + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST-1:PORT" name="federation-example-to-1"> + <federation> + <local-address-policy name="example-local-address-policy-1" max-hops="1"> + <include address-match="a.federated-address.#" /> + </local-address-policy> + </federation> + </amqp-connection> + <amqp-connection uri="tcp://HOST-2:PORT" name="federation-example-to-2"> + <federation> + <local-address-policy name="example-local-address-policy-2" max-hops="1"> + <include address-match="a.federated-address.#" /> + </local-address-policy> + </federation> + </amqp-connection> +</broker-connections> +---- + +By configuring each of the brokers with a `max-hops` value as above the message will be stopped from moving beyond the neighboring broker that receives it and no message looping occurs. + +=== Address Federation Ring + +Another less common configuration is a good demonstration of a case where using the `max-hops` settings is critical for preventing an infinite loop of message federation. This is also known as a ring and involves creating an address federation configuration that connects to a neighboring broker amongst three or more broker instances. In this scenario the configuration must set a `max-hops` value of N-1 where N is the number of brokers in the ring. + +image::images/amqp-federation-address-ring.png[] + +To achieve this configuration each broker would need to add a broker connection for its neighbor as follows. Note the `max-hops=4` in this case since there are five brokers in the diagram, this ensures a federated message never cross back through the originating broker and thus cannot create an message loop. + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST-1:PORT" name="federation-example-to-1"> + <federation> + <local-address-policy name="example-local-address-policy-1" max-hops="4"> + <include address-match="a.federated-address.#" /> + </local-address-policy> + </federation> + </amqp-connection> + </amqp-connection> +</broker-connections> +---- + +=== Hub and Spoke Address Federation + +Another configuration that allows consumers on brokers to read messages produced on any other broker peer is a hub and spoke configuration. In this configuration spoke brokers connect to a central hub and message traffic between the spokes passes through that hub broker which centralizes communication and allows the spokes to only need to know about one other broker, the hub broker. + +image::images/amqp-federation-address-hub-spoke.png[] + +This is configured on the spoke brokers much the same as dual address federation where a local and remote policy is configured both sides to listen for demand on the given address and federate messages when demand is detected. This setup allows the hub to now sense demand and federate from the spoke as needed and since each spoke mirrors this configuration the message can travel in either direction based on which spoke a consumer has subscribed to the federated address and which spoke a [...] + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HUB-HOST:PORT" name="federation-example"> + <federation> + <local-address-policy name="example-local-address-policy"> + <include address-match="a.federated-address.#" /> + </local-address-policy> + <remote-address-policy name="example-remote-address-policy"> + <include address-match="a.federated-address.#" /> + </remote-address-policy> + </federation> + </amqp-connection> +</broker-connections> +---- + +While this is not the only configuration that can create this setup it does represent a simple and flexible configuration where spoke can be added without making configuration changes on the hub. diff --git a/docs/user-manual/amqp-broker-connections.adoc b/docs/user-manual/amqp-broker-connections.adoc index 057b812054..540a0edd7d 100644 --- a/docs/user-manual/amqp-broker-connections.adoc +++ b/docs/user-manual/amqp-broker-connections.adoc @@ -429,7 +429,7 @@ IMPORTANT: To avoid confusion it is recommended that `address` `name` and `queue == Federation Broker federation allows the local broker to create remote receivers for addresses or queues that have local demand. -Conversely, the broker connection can send federation configuration to the remote broker causing it to create receivers on the local broker based on remote demand on an address or queue over this same connection. +Conversely, the broker connection can send connection scoped federation configuration to the remote broker causing it to create receivers on the local broker based on remote demand on an address or queue over this same connection. Add a `<federation>` element within the `<amqp-connection>` element to configure federation to the broker instance. The `<amqp-connection>` contains all the configuration for authentication and reconnection handling. @@ -461,97 +461,23 @@ The broker connection federation configuration consists of one or more policies </broker-connections> ---- -=== Local and remote address federation +=== Address federation -Local or Remote address federation configures the local or remote broker to watch for demand on addresses. -When demand exists it will create a consumer on the matching address on the opposing broker. -Because the consumers are created on addresses on the opposing broker, the authentication credentials supplied to the broker connection must have sufficient access to read from (local address federation) or write to the matching address (remote address federation) on the opposing broker. +Address federation can be thought of as full multicast over a set of loosely coupled brokers. Every message sent to address on `Broker-1` will be delivered to every queue bound to that address on that broker, but also will be delivered to the matching address on `Broker-2`, `Broker-3` ... `Broker-N` and all the queues bound to that address. -An example of address federation configuration is shown below. -Local and remote address policies have identical configuration parameters, only the policy element names are different. +.Address Federation +image::images/amqp-federation-address-concept.png[] -[,xml] ----- -<broker-connections> - <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> - <federation> - <local-address-policy name="example-local-address-policy" - auto-delete="false" - auto-delete-delay="1" - auto-delete-message-count="12" - max-hops="1" - enable-divert-bindings="true"> - <include address-match="local-address.#" /> - <exclude address-match="local-address.excluded" /> - </local-address-policy> - </federation> - </amqp-connection> -</broker-connections> ----- +For further details please see the xref:amqp-address-federation.adoc#address-federation[Address Federation] documentation. -name:: -The name of the policy, these names should be unique within a broker connection's federation policy elements. -include:: -The address-match pattern to use to match included addresses, multiple of these can be set. -If none are set all addresses are matched. -exclude:: -The address-match pattern to use to match excluded addresses, multiple of these can be set, or it can be omitted if no excludes are needed. -max-hops:: -The maximum number of hops that a message can perform to be federated. -Default value is `0` and will work for most simple federation deployments. -However, in certain topologies a higher value may be required to prevent looping. -auto-delete:: -For address federation, a durable queue is created on the broker from which messages are being federated. -Set this property to `true` to mark the queue for automatic deletion once the initiating broker disconnects and the delay and message count parameters are met. -This is a useful option if you want to automate the cleanup of dynamically-created queues. -The default value is `false`, which means that the queue is not automatically deleted. -auto-delete-delay:: -The amount of time in milliseconds after the initiating broker has disconnected before the created queue can be eligible for `auto-delete`. -The default value is `0` when option is omitted. -auto-delete-message-count:: -After the initiating broker has disconnected, the maximum number of messages allowed in the remote queue for the queue to be eligible for automatic deletion. -The default value is `0`. -enable-divert-bindings:: -Setting to `true` enables divert bindings to be listened-to for demand. -If a divert binding with an address matches the included addresses for the address policy, any queue bindings that match the forwarding address of the divert creates demand. The default value is `false`. +=== Queue federation -=== Local and remote queue federation +Queue federation offers a means of load balancing message queues across multiple broker instances. Messages sent to a queue on `Broker-1` will be consumed and sent to the matching queue on `Broker-2` if there is no local consumer available to consume the message. -Local or Remote queue federation configures the local or remote broker to watch for demand on queues and when demand exists it will create a consumer on the matching queue on the opposing broker. -Because the consumers are created on queues on the opposing broker, the authentication credentials supplied to the broker connection must have sufficient access to read from (local queue federation) or write to the matching queue (remote queue federation) on the opposing broker. +.Queue Federation +image::images/amqp-federation-queue-concept.png[] -An example of queue federation configuration is shown below. -Local and remote queue policies have identical configuration parameters, only the policy element names are different. - -[,xml] ----- -<broker-connections> - <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> - <federation> - <local-queue-policy name="example-local-queue-policy"> - <include address-match="#" queue-match="remote-queue" /> - <exclude address-match="excluded.#" queue-match="remote-queue-excluded" /> - </local-queue-policy> - </federation> - </amqp-connection> -</broker-connections> ----- - -name:: -The name of the policy. -These names should be unique within a broker connection's federation policy elements. -include:: -The queue-match pattern to use to match included queues. -Multiple of these can be set. -If none are set all queues are matched. -exclude:: -The queue-match pattern to use to match excluded queues. -Multiple of these can be set, or it can be omitted if no excludes are needed. -priority-adjustment:: -When federation consumers are created this value can be used to ensure that those federation consumers have a lower priority value than other local consumers on the same queue. -The default is `-1`. -include-federated:: -Controls if consumers on a queue which come from federation instances should be counted when observing a queue for demand, by default this value is `false` and federation consumers are not counted. +For further details please see the xref:amqp-queue-federation.adoc#queue-federation[Queue Federation] documentation. == Bridges diff --git a/docs/user-manual/amqp-federation-configuration-glossary.adoc b/docs/user-manual/amqp-federation-configuration-glossary.adoc new file mode 100644 index 0000000000..4c18bf77bf --- /dev/null +++ b/docs/user-manual/amqp-federation-configuration-glossary.adoc @@ -0,0 +1,240 @@ += AMQP Federation Configuration Glossary +:idprefix: +:idseparator: - +:docinfo: shared + +Federation configuration occurs in a broker connection element where a set of policies defines the desired address or queue federation and the locality of that federation (local or remote). A single federation instance is supported per broker connection element however each federation instance can contain multiple address and queue policy elements. Address and queue policy configuration exist for both the local and the remote side of the federation over the broker connection. An example [...] + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> + <federation> + <local-address-policy name="example-local-address-policy"> + <include address-match="local-address.#" /> + <exclude address-match="local-address.excluded" /> + </local-address-policy> + <local-queue-policy name="example-local-queue-policy"> + <include address-match="address" queue-match="local-queue" /> + </local-queue-policy> + <remote-address-policy name="example-remote-address-policy"> + <include address-match="remote-address" /> + </remote-address-policy> + <remote-queue-policy name="example-remote-queue-policy"> + <include address-match="#" queue-match="remote-queue" /> + <exclude address-match="excluded.#" queue-match="remote-queue-excluded" /> + </remote-queue-policy> + </federation> + </amqp-connection> +</broker-connections> +---- + +Local or Remote federation policies configure the local or remote broker to perform the federation operation over the defined connection but otherwise use the same configuration attributes and properties. The only difference is whether the federation occurs locally meaning local demand triggers consumers to be created towards the remote broker, or if remote demand triggers consumers to be created towards the local broker (local meaning the one that carries the configuration). For this re [...] + +Federation configuration can contain multiple policies of the same type (address or queue) along with a mix of types as seen in the configuration example above. This allows for policies with differing configurations for federating different addresses and queues within the same broker connection. It is important to note that the policy names must be unique within the federation configuration. The following configuration example demonstrates the mix of policy configurations within a single [...] + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> + <federation> + <local-address-policy name="example-local-address-policy-1" + auto-delete="true" + auto-delete-delay="30000" + auto-delete-message-count="0"> + <include address-match="a.local-address.#" /> + <exclude address-match="a.local-address.excluded" /> + </local-address-policy> + <local-address-policy name="example-local-address-policy-2" + auto-delete="false"> + <include address-match="b.local-address.#" /> + <exclude address-match="b.local-address.excluded" /> + </local-address-policy> + </federation> + </amqp-connection> +</broker-connections> +---- + +It is also supported to have only remote federation policies within a broker connection federation element. In this case the policy is sent across the connection to the remote broker and federation occurs on the remote for the life of that broker connection. + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> + <federation> + <remote-queue-policy name="example-remote-queue-policy"> + <include address-match="#" queue-match="remote-queue" /> + <exclude address-match="excluded.#" queue-match="remote-queue-excluded" /> + </remote-queue-policy> + </federation> + </amqp-connection> +</broker-connections> +---- + +=== Address federation configuration elements + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> + <federation> + <local-address-policy name="example-local-address-policy" + auto-delete="true" + auto-delete-delay="30000" + auto-delete-message-count="0" + max-hops="1" + enable-divert-bindings="true"> + <include address-match="local-address.#" /> + <exclude address-match="local-address.excluded" /> + <property key="amqpCredits" value="100" /> + <property key="amqpLowCredits" value="30" /> + <property key="addressReceiverIdleTimeout" value="0" /> + </local-address-policy> + </federation> + </amqp-connection> +</broker-connections> +---- + +The address policy element can be configured with the following set of attributes, or if the value is omitted the stated default value will be applied to the deployed policy. + +name:: +The name of the policy, these names should be unique within a broker connection's federation policy elements. +max-hops:: +The maximum number of hops that a message can perform to be federated. Default value is `0` and will work for most federation deployments. +However, in certain topologies where a loop is configured a higher value may be required to prevent infinite looping of messages. +auto-delete:: +For address federation, a durable queue is created on the broker from which messages are being federated. +Set this property to `true` to mark the queue for automatic deletion once the initiating broker disconnects and the delay and message count parameters are met. +This is a useful option if you want to automate the cleanup of dynamically-created queues. +The default value is `false`, which means that the queue is not automatically deleted. +auto-delete-delay:: +The amount of time in milliseconds after the initiating broker has disconnected before the created queue can be eligible for `auto-delete`. +The default value is `0` when option is omitted. +auto-delete-message-count:: +After the initiating broker has disconnected, the maximum number of messages allowed in the remote queue for the queue to be eligible for automatic deletion. The default value is `0`. +enable-divert-bindings:: +Setting to `true` enables divert bindings to be listened-to for demand. +If a divert binding with an address matches the included addresses for the address policy, any queue bindings that match the forwarding address of the divert creates demand. The default value is `false`. + +The following set of elements can be configured within the address policy to control the behavior of the deployed policy. For the policy to actually match any addresses on the target broker an include must be provided at a minimum. + +include:: +The address-match pattern to use to match included addresses, multiple of these can be set per policy. +exclude:: +The address-match pattern to use to match excluded addresses, multiple of these can be set, or it can be omitted if no excludes are needed. +property:: +A key / value property setting that allows for configuration options to be set either at the root federation level or within each federation policy that affects some defined behavior of the policy implementation. + +=== Queue federation configuration elements + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> + <federation> + <local-queue-policy name="example-local-queue-policy" priority-adjustment="-1"> + <include address-match="#" queue-match="remote-queue" /> + <exclude address-match="excluded.#" queue-match="remote-queue-excluded" /> + <property key="amqpCredits" value="0" /> + <property key="amqpPullConsumerCredits" value="1" /> + </local-queue-policy> + </federation> + </amqp-connection> +</broker-connections> +---- + +The queue policy element can be configured with the following set of attributes, or if the value is omitted the stated default value will be applied to the deployed policy. + +name:: +The name of the policy. +These names should be unique within a broker connection's federation policy elements. +priority-adjustment:: +When federation consumers are created this value can be used to ensure that those federation consumers have a lower priority value than other local consumers on the same queue. The default is `-1`. +include-federated:: +Controls if consumers on a queue which come from federation instances should be counted when observing a queue for demand, by default this value is `false` and federation consumers are not counted as local demand. + +The following set of elements can be configured within the queue policy to control the behavior of the deployed policy. + +include:: +The queue-match pattern to use to match included queues and enforce that a matched queue is also bound to a specific address. Multiple of the include elements can be set per policy, but if none are set all queues would be matched unless there is an exclude configured. +exclude:: +The queue-match pattern to use to match excluded queues that the policy should ignore. Multiple of these can be set, or it can be omitted if no excludes are needed. +property:: +A key / value property setting that allows for configuration options to be set either at the root federation level or within each federation policy that affects some defined behavior of the policy implementation. + +=== Federation policy property configuration index + +The following properties can be set either at the federation element level or set per-policy in order to apply different behaviors to different policies within the same federation configuration. When remote address or queue policies are configured the value of these properties are sent to the remote along with the policies to ensure both sides are operating with the same configurations. Certain value can also be set in the broker connection URI when noted if not set in any of the availab [...] + +|=== +| Name | Type | Description | Default + +| attach-timeout +| Integer +| Controls the timeout value for a federation receiver link attach to complete before the attach attempt is considered to have failed. The value is configured in seconds. +| 30 (Seconds) + +| amqpCredits +| Integer +| Defines the amount of credits to batch to an AMQP receiver link and the top up limit when sending more credit once the credits are determined to be running low. This value will be read from the broker connection URI if one was specified there and not set in the federation policy.. +| 1000 (broker default) + +| amqpLowCredits +| Integer +| A low water mark for receiver credits that indicates more should be sent to top it up to the original credit batch size. This value will be read from the broker connection URI if one was specified there and not overridden in the federation policy. +| 300 (broker default) + +| amqpPullConsumerCredits +| Integer +| Defines the amount of credits to batch to an AMQP receiver link and the top up value when sending more credit once the broker has capacity available for them. In order for this value to take affect the `amqpCredits` value must be set to zero. If `amqpCredits` ist set to zero and this option is not configured the default value is used. +| 100 + +| minLargeMessageSize +| Integer +| Used to configure the value to use when considering if a message is a large message for federation consumers. This value will be read from the broker connection URI if one was specified there and not overridden in the federation policy. +| Broker default + +| ignoreAddressBindingFilters +| Boolean +| Used to configure if federation address consumers should filter messages using the filters defined on individual matching address bindings. This value can be used to reduce the amount of messages that are federated for an address if the local consumers use filters but must be enabled with care as it can easily lead to duplicate messages being dispatched from the remote as each unique address binding filter creates a remote consumer with that matching filter. If any filters overlap in t [...] +| true + +| ignoreQueueConsumerFilters +| Boolean +| Used to configure if federation queue consumers should filter messages using the filters defined on individual queue subscriptions. This can be used to prevent multiple subscriptions on the same queue based on local demand with differing subscription filters but does imply that message that don't match those filters would be federated to the local broker. +| false + +| ignoreQueueConsumerPriorities +| Boolean +| Used to control if federation queue consumers should apply a consumer priority offset based on the subscription priority or should use a singular priority offset based on policy configuration. This can be used to prevent multiple subscriptions on the same queue based on local demand with differing consumer priorities but does imply that care needs to be taken to ensure remote consumers would still normally have a higher priority value than the configured default priority offset. This v [...] +| true + +| amqpUseModifiedForTransientDeliveryErrors +| Boolean +| Used to configure how a federation receiver should respond to delivery errors indicating that an address is full and cannot accept messages at this time. By default we will send Modified outcomes with the delivery failed value set to true such that the remote will deliver the message again after incrementing the delivery count of the message. Disabling this causes the Rejected outcome to be sent instead which is a terminal outcome which results in message being dropped and possibly sen [...] +| true + +| amqpLinkQuiesceTimeout +| Integer +| Defines the time in milliseconds that a receiver will wait before considering a pending quiesce timeout to have failed and should close the link. This option can be used to override the value specified on the connector URI to allow federation to operate with a different default. This timeout applies to AMQP links that are being drained due to a transient delivery error such as the address being full and the address policy is set to send a failed response to the remote sender. If the dr [...] +| Broker default (10 minutes) + +| amqpDrainOnTransientDeliveryErrors +| Boolean +| Used to control if a federation receiver should drain the link credit when a transient delivery error such as the address being full occurs. This option can be used to override the value specified on the connector URI to allow federation to operate with a different default. +| Broker default (true). + +| receiverQuiesceTimeout +| Integer +| When a federation receiver link is being drained due to removal of local demand this timeout value enforces a maximum wait for drain and processing of in-flight messages before the link is forcibly terminated with the assumption that the remote is no longer responding. +| 60s + +| addressReceiverIdleTimeout +| Integer +| When a federation address receiver link has been successfully drained after demand was removed from the federated resource, this value controls how long the link can remain in an attached but idle state before it is closed. This can allow for quicker restoration of federated messages if the demand on a queue is removed and re-added quickly as the link is drained of credit but not immediately closed. For address receivers the default is kept short to prevent holding broadcast messages o [...] +| 5s + +| queueReceiverIdleTimeout +| Integer +| When a federation queue receiver link has been successfully drained after demand was removed from the federated resource, this value controls how long the link can remain in an attached but idle state before it is closed. This can allow for quicker restoration of federated messages if the demand on a queue is removed and re-added quickly as the link is drained of credit but not immediately closed. +| 60s diff --git a/docs/user-manual/amqp-queue-federation.adoc b/docs/user-manual/amqp-queue-federation.adoc new file mode 100644 index 0000000000..70dd5df763 --- /dev/null +++ b/docs/user-manual/amqp-queue-federation.adoc @@ -0,0 +1,107 @@ += Queue Federation +:idprefix: +:idseparator: - +:docinfo: shared + +Queue federation offers a means of load balancing message queues across multiple broker instances. Messages sent to a queue on `Broker-1` can be consumed and sent to the matching queue on `Broker-2` via a federation consumer if there is no local consumer available to consume the message. + +Queue federation creates consumers that consume messages from the matching queue on the remote peer when demand is detached on the local queue that matches a specific queue federation policy. The federation consumer acts as an addition consumer on the target remote queue competing with consumers on that remote queue for messages. Federation queue consumers can be configured to always have lower priority than the local queue consumers and also can be configured to only pull messages from [...] + +image::images/amqp-federation-queue-concept.png[] + +Queue federation can be configured operate in a single direction between peers or can configured to operate in both directions between peers allowing message to flow in both directions delivering messages to consumers on either broker that were sent to the matching queue on the opposing broker instance. When configured for dual federation messages can migrate back and forth between brokers depending on capacity and available local consumers, this back and forth message flow can be reduce [...] + +== Common Queue Federation Topologies + +The sections that follow discuss some common queue federation topologies and some configuration considerations specific to those topologies. + +=== Simple Queue Load Balance + +The simplest configuration of Queue federation involves a two broker setup where `broker-1` creates a federation consumer to `broker-2` when demand is present on a federated queue. The messages sent by a producer on `broker-2` are shared between any local consumer and the federation consumer. + +image::images/amqp-federation-queue-simple-lb.png[] + +This configuration can be setup by adding a federation policy on `broker-1` that configure the queue to federate and a connection URI pointing to the opposing broker. Note that this configuration adds properties to configure the federation queue consumer to pull messages as opposed to the default which allows messages to flow regardless of local queue backlogs, this helps to prevent pulling messages from the remote broker unless the local broker has a shortage which helps to prevent star [...] + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> + <federation> + <local-queue-policy name="example-local-address-policy"> + <include address-match="#" queue-match="a.federated-queue" /> + <property key="amqpCredits" value="0" /> + <property key="amqpPullConsumerCredits" value="1" /> + </local-queue-policy> + </federation> + </amqp-connection> +</broker-connections> +---- + +While this configuration provides basic sharing of messages between brokers it can result in messages being stuck on `broker-1` if the consumer goes down as messages cannot flow back to `broker-2` to be be read by a consumer there which is why the next topology is often deployed. + +=== Bi-directional Queue Federation + +In a bi-directional queue federation configuration both brokers are configured to federate a given queue between them allowing messages to flow in either direction. This configuration allows message that have been federated to one broker to flow back to the other if local demand is removed which can prevent stuck messages. + +image::images/amqp-federation-queue-bi-directional.png[] + +This configuration can be setup by adding a broker connection element to each of the broker instances with the same local queue federation policy or could be defined on only one broker using both a local and remote queue federation policy as shown below. + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> + <federation> + <local-queue-policy name="example-local-queue-policy"> + <include address-match="#" queue-match="a.federated-queue" /> + </local-queue-policy> + <remote-queue-policy name="example-remote-queue-policy"> + <include address-match="#" queue-match="a.federated-queue" /> + </remote-queue-policy> + <property key="amqpCredits" value="0" /> + <property key="amqpPullConsumerCredits" value="10" /> + </federation> + </amqp-connection> +</broker-connections> +---- + +This configuration would result in only a single connection between the two brokers. Each policy will be configured by the shared properties of the federation element to use a pull consumer that batches ten messages at a time whenever the target queue has no pending messages already waiting to be read. + +=== Queue Federation with Intermediary Brokers + +In some use cases a federated queue may need to cross through an intermediary broker before reaching a consumer. For example consider a three broker chain where messages produced on `broker-1` must pass through `broker-2` before reaching a consumer on `broker-3`. + +image::images/amqp-federation-queue-with-hop.png[] + +To accomplish this the federation queue policy for the intermediary broker must have additional configuration to allow for federation of messages when no local demand is present but a federation consumer is. Normally federation queue policies do no consider a federation queue binding on a local queue as demand to avoid creating loops. To disable that behavior the policy supports the attribute `include-federated` which defaults to `false`. The XML configuration for `broker-2` is below tha [...] + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> + <federation> + <local-queue-policy name="example-local-queue-policy" include-federated="true"> + <include queue-match="a.federated-queue" /> + </local-queue-policy> + </federation> + </amqp-connection> +</broker-connections> +---- + +The configuration on `broker-3` looks essentially the same as that on `broker-2` however the `include-federated` attribute can be omitted. + +[,xml] +---- +<broker-connections> + <amqp-connection uri="tcp://HOST:PORT" name="federation-example"> + <federation> + <local-queue-policy name="example-local-queue-policy"> + <include queue-match="a.federated-queue" /> + </local-queue-policy> + </federation> + </amqp-connection> +</broker-connections> +---- + +With this configuration in place messages produced on `broker-1` will be federated to `broker-2` and then on to `broker-3` whenever a consumer is attached to `broker-3`. It should be noted however that in a configuration such as this that messages can be left on `broker-2` if they were arriving there at the same time that the consumer on `broker-3` had gone away and the federation link between `broker-3` and `broker-2` was gearing down. \ No newline at end of file diff --git a/docs/user-manual/images/amqp-federation-address-bi-directional.png b/docs/user-manual/images/amqp-federation-address-bi-directional.png new file mode 100644 index 0000000000..47a43061d8 Binary files /dev/null and b/docs/user-manual/images/amqp-federation-address-bi-directional.png differ diff --git a/docs/user-manual/images/amqp-federation-address-concept.png b/docs/user-manual/images/amqp-federation-address-concept.png new file mode 100644 index 0000000000..0982a93566 Binary files /dev/null and b/docs/user-manual/images/amqp-federation-address-concept.png differ diff --git a/docs/user-manual/images/amqp-federation-address-hub-spoke.png b/docs/user-manual/images/amqp-federation-address-hub-spoke.png new file mode 100644 index 0000000000..ccb1f826a8 Binary files /dev/null and b/docs/user-manual/images/amqp-federation-address-hub-spoke.png differ diff --git a/docs/user-manual/images/amqp-federation-address-mesh.png b/docs/user-manual/images/amqp-federation-address-mesh.png new file mode 100644 index 0000000000..38316b678f Binary files /dev/null and b/docs/user-manual/images/amqp-federation-address-mesh.png differ diff --git a/docs/user-manual/images/amqp-federation-address-ring.png b/docs/user-manual/images/amqp-federation-address-ring.png new file mode 100644 index 0000000000..5cdc9649a2 Binary files /dev/null and b/docs/user-manual/images/amqp-federation-address-ring.png differ diff --git a/docs/user-manual/images/amqp-federation-queue-bi-directional.png b/docs/user-manual/images/amqp-federation-queue-bi-directional.png new file mode 100644 index 0000000000..003714454d Binary files /dev/null and b/docs/user-manual/images/amqp-federation-queue-bi-directional.png differ diff --git a/docs/user-manual/images/amqp-federation-queue-concept.png b/docs/user-manual/images/amqp-federation-queue-concept.png new file mode 100644 index 0000000000..091c01f288 Binary files /dev/null and b/docs/user-manual/images/amqp-federation-queue-concept.png differ diff --git a/docs/user-manual/images/amqp-federation-queue-simple-lb.png b/docs/user-manual/images/amqp-federation-queue-simple-lb.png new file mode 100644 index 0000000000..7f199c9f4f Binary files /dev/null and b/docs/user-manual/images/amqp-federation-queue-simple-lb.png differ diff --git a/docs/user-manual/images/amqp-federation-queue-with-hop.png b/docs/user-manual/images/amqp-federation-queue-with-hop.png new file mode 100644 index 0000000000..ab7f7d76c9 Binary files /dev/null and b/docs/user-manual/images/amqp-federation-queue-with-hop.png differ diff --git a/docs/user-manual/index.adoc b/docs/user-manual/index.adoc index c7f1605d39..dee1ebd773 100644 --- a/docs/user-manual/index.adoc +++ b/docs/user-manual/index.adoc @@ -72,8 +72,8 @@ image::images/activemq-logo.png[align="center"] ** xref:amqp-broker-connections.adoc#amqp-server-connections[Server Connections] ** xref:amqp-broker-connections.adoc#mirroring[Mirroring] ** xref:amqp-broker-connections.adoc#federation[AMQP Federation] -*** xref:amqp-broker-connections.adoc#local-and-remote-address-federation[Address Federation] -*** xref:amqp-broker-connections.adoc#local-and-remote-queue-federation[Queue Federation] +*** xref:amqp-broker-connections.adoc#address-federation[Address Federation] +*** xref:amqp-broker-connections.adoc#queue-federation[Queue Federation] ** xref:amqp-broker-connections.adoc#bridges[AMQP Bridges] *** xref:amqp-broker-connections.adoc#bridging-from-remote-addresses-and-queues[Bridging From a remote] *** xref:amqp-broker-connections.adoc#bridging-to-remote-addresses-and-queues[Bridging To a remote] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
