[
https://issues.apache.org/jira/browse/ARTEMIS-3243?focusedWorklogId=633053&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-633053
]
ASF GitHub Bot logged work on ARTEMIS-3243:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 03/Aug/21 16:59
Start Date: 03/Aug/21 16:59
Worklog Time Spent: 10m
Work Description: clebertsuconic commented on a change in pull request
#3633:
URL: https://github.com/apache/activemq-artemis/pull/3633#discussion_r681941949
##########
File path: docs/user-manual/en/amqp-broker-connections.md
##########
@@ -32,121 +32,145 @@ To define an AMQP broker connection, add an
`<amqp-connection>` element within t
- `password`: Password with which to connect to the endpoint (this is an
optional argument)
- `retry-interval`: Time, in milliseconds to wait before retrying a connection
after an error. The default value is `5000`.
- `reconnect-attempts`: default is -1 meaning infinite
-- `auto-start` : Should the broker connection start automatically with the
broker. Default is `true`. If false you need to call a management operation to
start it.
+- `auto-start` : Should the broker connection start automatically with the
broker. Default is `true`. If false it is necessary to call a management
operation to start it.
-*Notice:* If you disable auto-start on the broker connection, the start of the
broker connection will only happen after the management method
`startBrokerConnection(connectionName)` is called on the ServerController.
+*Notice:* If auto-start is disabled on the broker connection, the start of the
broker connection will only happen after the management method
`startBrokerConnection(connectionName)` is called on the ServerController.
-*Important*: The target endpoint needs permission for all operations that you
configure. Therefore, If you are using a security manager, ensure that you
perform the configured operations as a user with sufficient permissions.
+*Important*: The target endpoint needs permission for all operations that
configured. Therefore, If a security manager is being used, ensure to perform
the configured operations with a user with sufficient permissions.
# AMQP Server Connection Operations
-The following types of operations are supported on a AMQP server connection:
+The following types of operations are supported on an AMQP server connection:
+* Mirrors
+ * The broker uses an AMQP connection to another broker and duplicates
messages and sends acknowledgements over the wire.
* Senders
- * Messages received on specific queues are transferred to another endpoint
+ * Messages received on specific queues are transferred to another endpoint.
* Receivers
- * The broker pulls messages from another endpoint
+ * The broker pulls messages from another endpoint.
* Peers
- * The broker creates both senders and receivers on another endpoint that
knows how to handle them. Currently, this is implemented by Apache Qpid
Dispatch.
-* Mirrors
- * The broker uses an AMQP connection to another broker and duplicate
messages and sends acknowledgements over the wire.
+ * The broker creates both senders and receivers on another endpoint that
knows how to handle them. This is currently implemented by Apache Qpid Dispatch.
+
+# Reconnecting and Failover
-## Senders and Receivers
-It is possible to connect an ActiveMQ Artemis broker to another AMQP endpoint
simply by creating a sender or receiver broker connection element.
+It is possible to determine how reconnection will happen on a broker
connection.
-For a `sender`, the broker creates a message consumer on a queue that sends
messages to another AMQP endpoint.
+These are the attributes are available on amqp-connection XML element:
+- reconnect-attempts: default as -1 (infinite). How many attempts will be done
after a failed connection
+- retry-interval: default as 5000, in milliseconds, the wait between each
retry in connections.
-For a `receiver`, the broker creates a message producer on an address that
receives messages from another AMQP endpoint.
+It is also possible to specify alternate hosts on a broker connection by
appending a comma separated list after a # at the end of the URI.
+The broker connection would keep trying on the alternate list until one of the
targets is available to connect. Example:
-Both elements work like a message bridge. However, there is no additional
overhead required to process messages. Senders and receivers behave just like
any other consumer or producer in ActiveMQ Artemis.
+
+Figure 2. Broker Connection Disaster and Recovery.
-You can configure senders or receivers for specific queues. You can also match
senders and receivers to specific addresses or _sets_ of addresses, using
wildcard expressions. When configuring a sender or receiver, you can set the
following properties:
+The previous example portrays a case of connection failure towards ServerA.
The system would try to connect between serverA, backupA, and backupB until it
successfully connects to any of these nodes.
-- `address-match`: Match the sender or receiver to a specific address or
__set__ of addresses, using a wildcard expression
-- `queue-name`: Configure the sender or receiver for a specific queue
+## Mirroring
+This is probably the main use case for Broker Connections and the main reason
broker connections were developed in the first place.
+The idea of mirroring is to send every event that happened on a broker towards
another broker, without blocking any operations from producers and consumers as
fast as possible.
+It can be used for Disaster and Recovery, and depending on the requirements
even for failing over the data.
-Some examples are shown below.
+The following events are sent through mirroring:
-Using address expressions:
-```xml
-<broker-connections>
- <amqp-connection uri="tcp://MY_HOST:MY_PORT" name="my-broker">
- <sender address-match="queues.#"/>
- <!-- notice the local queues for remotequeues.# need to be created on
this broker -->
- <receiver address-match="remotequeues.#"/>
- </amqp-connection>
-</broker-connections>
+* Message sending
+ * Messages sent in one broker will be "replicated" to the target broker.
+* Message acknowledgement
+ * Messages removed through acks in one broker will be sent towards the
target broker.
+ * Notice if the message is pending on a consumer on the target mirror, the
ack will not succeed and the message might be delivered on both ends.
+* Queue and address creation.
+* Queue and address deletion.
-<addresses>
- <address name="remotequeues.A">
- <anycast>
- <queue name="remoteQueueA"/>
- </anycast>
- </address>
- <address name="queues.B">
- <anycast>
- <queue name="localQueueB"/>
- </anycast>
- </address>
-</addresses>
-```
-Using queue names:
+Add a `<mirror>` element within the `<amqp-connection>` element to configure
it.
+
+### Mirror configuration:
+
+The following optional arguments can be utilized:
+
+* `queue-removal`: Specifies whether a queue- or address-removal event is
sent. The default value is `true`.
+* `message-acknowledgements`: Specifies whether message acknowledgements are
sent. The default value is `true`.
+* `queue-creation`: Specifies whether a queue- or address-creation event is
sent. The default value is `true`.
+
+An example of a mirror configuration is shown below:
```xml
<broker-connections>
- <amqp-connection uri="tcp://MY_HOST:MY_PORT" name="my-broker">
- <receiver queue-name="remoteQueueA"/>
- <sender queue-name="localQueueB"/>
- </amqp-connection>
+ <amqp-connection uri="tcp://MY_HOST:MY_PORT" name="my-mirror">
+ <mirror queue-removal="true" queue-creation="true"
message-acknowledgements="true" source-mirror-address="myLocalSNFMirrorQueue"/>
+ </amqp-connection>
</broker-connections>
+```
-<addresses>
- <address name="remotequeues.A">
- <anycast>
- <queue name="remoteQueueA"/>
- </anycast>
- </address>
- <address name="queues.B">
- <anycast>
- <queue name="localQueueB"/>
- </anycast>
- </address>
-</addresses>
+
+### Store and Forward Queue
+
+Mirror events are always stored on a local queue prefixed as
"$ACTIVEMQ_ARTEMIS_MIRROR_" concatenated with broker connection's name.
+
+So, in the following configuration mirror events will be stored on a queue
named "$ACTIVEMQ_ARTEMIS_MIRROR_my-mirror".
+
+```xml
+<broker-connection>
+ <amqp-connection uri="tcp://brokerB:5672" name="brokerB">
+ <mirror/>
+ </amqp-connection>
+</broker-connection>
```
-*Important:* You can match a receiver only to a local queue that already
exists. Therefore, if you are using receivers, make sure that you pre-create
the queue locally. Otherwise, the broker cannot match the remote queues and
addresses.
-*Important:* Do not create a sender and a receiver to the same destination.
This creates an infinite loop of sends and receives.
+These messages are then transferred to brokerB:5672. A link with the address
name $ACTIVEMQ_ARTEMIS_MIRROR_brokerB will be created towards brokerB. If there
is a security manager configured, security roles must be provided to the user
on the broker connection.
+
+Notice the queue $ACTIVEMQ_ARTEMIS_MIRROR_brokerB will not physically exist on
brokerB and there won't be a way to find it on the administrator console. The
broker will treat these messages accordingly as mirror events and will execute
the appropriate operations at the target broker.
+
+### Pre Existing Messages
+The brokers will only send messages from the point in time the mirror was
configured. Previously existing messages will not be forwarded to other brokers.
+
+## Dual Mirror (Disaster and Recovery option)
+
+ActiveMQ Artemis supports automatic failback mirroring. Every sent message and
every acknowledgement is asynchronously replicated to the mirrored broker.
+
+On the following diagram, there will be two servers called DataCenter1, and
DataCenter2. In order to have a dual mirror configuration, it is necessary is
to add the mirror broker connection on each broker.xml:
+
+
+Figure 2. Broker Connection Disaster and Recovery.
+The broker connection will replicate sends and acknowledgements no matter
where they originated. If messages are sent on DC1 (DataCenter1) these will be
automatically transitioned to DC2 (DataCenter2). Messages Acknowledged on DC2
will be transitionally back to DC1.
+The only exception to that rule would be if there were consumers with pending
messages on any server. It is recommended to not have active consumers on both
servers.
+
+## Example
+There is an example as part of the distribution showing dual broker
configuration (or disaster and recovery) under
./examples/features/broker-connection/disaster-recovery.
+
+On the example two brokers are configured to mirror each other and whatever
happens in one broker is immediately copied over to the other broker.
## Peers
-The broker can be configured as a peer which connects to the [Apache Qpid
Dispatch Router](https://qpid.apache.org/components/dispatch-router/) and
instructs it the broker it will act as a store-and-forward queue for a given
AMQP waypoint address configured on the router. In this scenario, clients
connect to a router to send and receive messages using a waypointed address,
and the router routes these messages to or from the queue on the broker.
+The broker can be configured as a peer which connects to the [Apache Qpid
Dispatch Router](https://qpid.apache.org/components/dispatch-router/) and
instructs it that the broker will act as a store-and-forward queue for a given
AMQP waypoint address configured on the router. In this scenario, clients
connect to a router to send and receive messages using a waypointed address,
and the router routes these messages to or from the queue on the broker.
Review comment:
good idea.. will do
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 633053)
Time Spent: 35h 20m (was: 35h 10m)
> Enhance AMQP Mirror support with dual mirror
> --------------------------------------------
>
> Key: ARTEMIS-3243
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3243
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Affects Versions: 2.17.0
> Reporter: Clebert Suconic
> Assignee: Clebert Suconic
> Priority: Major
> Fix For: 2.18.0
>
> Time Spent: 35h 20m
> Remaining Estimate: 0h
>
> at the current Mirror version, we can only mirror into a single direction.
> With this enhancement the two (or more brokers) would be connected to each
> other, each one having its own ID, and each one would send updates to the
> other broker.
> The outcome is that if you just transferred producers and consumers from one
> broker into the other, the fallback would be automatic and simple. No need to
> disable and enable mirror options.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
